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Study Group on the Conditions of Excellence in American 
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a document to summarize and organize the many available 
approaches for defining the outcomes of college arid university 
experiences. Because of their commitment to institutional assess- 
ment, members of the Study Group also were interested in dis- 
covering promising approaches to using assessment information 
to improve curriculum, instruction, and management. Prepara- 
tion of the Study Group paper— originally titled "Dimensions of 
Excellence in Pbst secon dary Education"— was supported by a 
grant from the W. K. Kellogg Foundation. Recornmendations on 
the need for institutional assessment and feedback became inter 
gral parts of the Study Group's report, Involvement m Learning, 
which was issued by the Department of Education in October 
1984. • / 
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Introduction 



Calls for improving the quality of our colleges and universities 
have been increasingly insistent of late. Following hard 
upon recent inquiries about the condition of elementary and 
secondary education have been numerous attempts to assess both 
the objectives and the effect of postsecondary education. More 
importantly, these assessments are accompanied by rising levels / 
of public uncertainty about the place and worth of higher educa- 
tion in American society. After several decades of unbroken 
confidence and unprecedented expansion, higher education is 
now being called upon to demonstrate that it has been making a 
difference. 

Thus has begun a growing movement toward making colleges 
and universities— particularly in the public sector— more expUddy 
responsible for demonstrating their educational effectiveness. 
More and more states, for example, are requiring institutions to 
engage in formal processes of academic program review (Barak 
1982) as concrete evidence of program quality. Furthermore, some 
states have initiated processes for assessing actual educational 



, 1 



ERIC 



10 



Peter Ewell 



outcomes— both for assuring ixnpetence among graduating 
students and for fewarding institutions that caft demonstrate 
effectiveness. Finally, One major recommendation of a recent 
national panel Op attaining excellence in higher education is for 
colleges and universities to explicitly assess the impacts thfey^re 
having on students (National Institute of Education 1984). 

In the face of such calls for improvement, the fc response of 
higher education has been, at best, ambiguous. Many educators 
argue that the very nature of the higher-education enterprise 
effectively precludes improvement through increased external 
accountability. Such positions rely implicitly .on the traditional 
decentralized, self-policing character of the academy to make war- 
ranted changes. Opposing positions call for establishing common 
external standards of accountability. These positions hotel that 
the academy's traditiqnal selfcorrective mechanisms, if they ever 
existed, have broken down. Such positions maintain that higher 
education's greatest current problem is a crisis of confidence 
among its constituents, and that the traditional assurances will 
no longer suffice, 

v £uch tensions are, of course, not unfamiliar in die develop- 
ment of social institutions. Writing 150 years ago in the preface to 
Democracy in America, Alexis de Tocqueville viewed with rising 
concerp the broadening and leveling effects of increased social 
equality and access. Broadened participation, he looted, brought 
with it increased general prosperity; its leveling tendencies, 
however, entailed inevitable threats to quality in many areas- 
most particularly education, the arts, and public life. As a result 
of this tension, Tocqueville reports that his inquiry "has been 
conducted under the impure of a kind of religious dread," a 
feeling arising both hon\ the seeming inevitability of the process 
to which he was a witness ami from his profound doubts about its 
probable consequences. 

Calls to increase accountability for educational outcomes 
should provoke in us analogous feelings of "religious dread." 
Higher education, too, has undergone profound changes in its 
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structure and characteristics over the past four decades {National 
Institute of Education 1984). Institutions have become larger, 
more specialized, and more public. Students have become more 
Varied in background, preparation, and patterns of attendance. 
Curricula have evolved from the traditional liberal arts to a broad 
array of occupational, professional, and special-purpose programs. 
As a result of this diversity, the pattern of higher -education out- 
comes has become almost bewilderingly complex. Thus/calls for 
accountability stem, at least in part, from a simple desire on the 
part of those who support higher education to understand what 
has.oCcurred, 

On the one hand; such calls for accountability are a healthy 
development. In an era of constrained resources and many alter- 
native opportunities for social investment, higher education can 
no longer affora to base its claims to resources on the seif-evidence 
of its benefits* Indeed, as institutions, students, and curricula * 
have changed, the actual nature of these benefits has .become 
increasingly murky. Traditional proponents of higher education 
see such benefits from within a time-honored perspective of broad 
liberal education— a heritage dominated by well-established 
models of the small college and the comprehensive research uni- 
versity. Consequently, these traditional proponents view the 
recent proliferation of community colleges and vocational institu- 
tions with alarm. Newer advocates of postsecondary education 
see its benefits in terms of their ability to answer society's need for ^ 
trained manpower ip critical fields.. As a Result, they view the 
maintenance of many traditional institutions as an expensive 
luxury. The situation has not been helped by the fact that the case 
for higher education has, in its expansionist past, been founded 
upon both positions, and has therefore involved some rather 
grandiose claims of effectiveness. What we are now increasingly 
being asked to demonstrate is nothing more than that for which 
we in the past have had the hubris to claim credit. Unpleasant as 
it may be, increased external accountability may force us to make 
necessary choices among alternative missions and priorities. 
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On the oti/cr hand, positions that stress the traditional 
autonomy of the academic enterprise are equally important. W: 
have learned repeatedly that the practices of teaching and 
scholarly inquiry are fragile indeed. To be effective, both require 
an open, participatory environment. Both operate best in the 
context of a decentralized and "values oriented" institution, one 
that can isolate itself somewhat from a wider, more instrumental 
environment. And both flourish only under situations where risk 
taking is seen as normal and is rewarded. Conformity to external 
standards, at whatever level such standards are developed or 
applied, is, in this context, an alien and a counterproductive 
activity. 

This is by no means to say that higher education ought there- 
fore to be unaccountable by any standards. The maintenance of 
standards within fndividual disciplines constitutes the core of the 
academic enterprise. Indeed, a first step in creating and maintain- 
ing e \!.. «il accountability should be founded upon this fact. 
Iteerr. t ndard setting and evaluation, a process familiar to us as 
practicing teachers and scholars, should be made both more 
generally recognized and more explicit in our institutions. 

If there is one generalization that can be made about the 
academic community, it is that we place unusual value on acquir- 
ing information and using it for social and individual improve- 
ment. Much of the historic rhetoric of scholarship, for example, 
emphasizes the role of the university in creating and maintaining 
society's store of knowledge. Furthermore, such rhetoric asserts 
thai this role is not self-fulfilling. Rather, it is undertaken to 
improve society's capacity to become more self-regarding and self- 
improving, to encourage society to recognize its strengths and 
correct its deficiencies. At the same time, an equally historic aim 
of higher education—particularly liberal education— is to produce 
these same self-regarding and setf^mproving qualities in individual 
learners. Reverence for exact information and its application 
to generate new knowledge and to improve practice are those 
qualities, above all, that we wish to instill in our students. They 
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are also precisely those qualities we seek to exemplify and reward 
in our disciplinary lives. 

Given that we have constructed a culture in higher education 
that appears so publicly to value information, it is surprising how 
little of it we tend to have about ourselves. And more surprising is 
the fact that what we do know about what works and what does 
not in particular colleges and universities has had but little effect 
on actual teaching and administration. The reasons for this ap- 
parent contradiction are many, some quite legitimate* The central 
theme of this Executive Overview is that to achieve excellence in 
the diverse activities currently comprising postsecondary educa- 
tion, wt^must create explicit, institution^specific mechanisms for 
regularly assessing the degree to which we are in fact attaining our 
collective goals. 

Such mechanisms are fast becoming hallmarks of what can be 
termed the self-regarding institution. Like the traditional small 
college, the self-regarding institution is aware of its distinctive- 
ness, its purposes, and its strengths and its deficiencies- Further- 
more, like the small college, it has ways of structuring a dialogue 
about itself that is carried on by all of its members. Most 
important of all, discussions of effectiveness in the self-regarding 
institution are informed discussions. Indeed, they are based upon 
explicit and available collective information about what students 
at the institution ait experiencing, and information about the 
linkages between various aspects of the institution's program and 
particular aspects of student growth and development. 

In most cases, however, current modes of organization in 
higher education continue little changed from the time when 
these impacts and linkages were directly observable by the 
academic community. In the environment of the small, private 
liberal-arts college or the elite research uni verity, a network of 
shared values and implicit communication tends automatically to 
keep issues of educational practice— its ends and its effectiveness— 
at the forefront of institutional discussion. In such a context, 
there is little need for explicit, structured information about how 
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the institution is faring. And indeed, in some of our best institu- 
tions these processes of value building and tacit communication 
remain vigorous and effective. 

For the majority of current postsecopdary institutions, 
however, these traditional self assessment mechanisms are not 
present. Not, despite recent calls to return to them (Martin 1982), 
are such mechanisms operationally feasible. The large public 
university, regional service institution, or community college is 
simply too big and too multicultural, inhabited by too many - 
student clienteles, and responsible to too many diverse constitu- 
encies to allow such implicit mechanisms for self-assessment. The 
challenge to administration in such situations is to create explicit, 
information-based structures of incentives and accountability to 
replace our more traditional implicit methods of self-assessment 
and self-improvement. 

Such systems can revitalize our institutions in much the same 
manner as traditional academic dialogue* Certainly, they must be 
constructed to be consistent with local institutional values and 
sensibilities. But they also haw the potential to reassure critical 
external constituencies— increasingly dissatisfied with higher 
education's silence on the question of effectiveness— that we are 
demonstrably concerned with achieving institutional excellence 
and that we have visibly committed ourselves to its assessment 
and improvement. 

Still, the desired ends of postsecondary education remain 
diverse and largely incommensurable. This poses an obstacle to 
establishing outcomes-oriented assessment mechanisms. Clearly, 
different kinds of higher-education institutions are in vastly 
different businesses and ought, therefore, to be held accountable 
for different things. Equally clear, however, is the demand that all 
institutions produce certified graduates with an assured level of 
basic or professional skill. Both objectives are legitimate, but they 
are difficult to discuss because as yet we have no common 
language that systematically distinguishes the several dimensions 
of postsecondary educational outcomes. In practice, this causes 
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some important misunderstandings. Is "educational outcome," 
for example, to refer tQ actual changes in student knowledge or 
abilities resulting from the learning process, regardless of the 
ultimate knowledge or skill level attained? Or is it rather to refer 
to the actual level of attainment it the end of the process, 
regardless of the level at which the student began? The current 
controversy over "value added" assessment is generated in large 
part by the conviction that we must judge institutional quahcy in 
terms of one of these criteria but not both. \tk could avoid nvtch 
of this confusion by recognizing that both notions are appro- 
priate, but for different analytical and assessment purposes. 

Equally daunting for institutions attempting to establish 
comprehensive self-assessment programs are the multiple, often 
highly complex, ways educational researchers heve chosen to 
structure the particular outcomes of postsecondary education. 
Each such structure has its virtues. Each provides a set of dimen- 
sions or categories well suited to a particular type of institution or 
method of analysis. But the majority of such conceptual schemes 
are relatively inaccessible and of limited use to the practicing 
academic administrator. 

The objective of part 1 of this book, therefore, is to sum- 
marize and organize these conceptual efforts and present them in 
such a way that some clear institutional choices can be made. An 
administrator needs tb know which of many possible outcomes 
dimensions are important and where, as a result, to focus an 
institutional-assessment effort. 

Establishing ruch an effort, however, requires a substantial 
reorientation of the way institutions of higher education typically 
do business. As in the elementary or secondary setting, the pro- 
cess of producing better "outputs" from higher education is a • 
function of the classroom, of the students who inhabit it, of 
the curriculum, and of the leaning environment. Vk in higher 
education certainly know less- than we should about the explicit 
linkages between such factors and particular kinds of student 
learning outcomes. 
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But we often know more than act upon. What is generally 
lacking in postsecondary classrooms is not intellectual talent 
but a consistent set of institutional incentives to use this talent 
effectively. Creating such a set of institutional incentives is an 
administrative task. It is not enough for us to know more about 
the probable linkages between learning factors and learning out- 
comes. To achieve excellence in the fragmented, organizationally 
conservative setting of the college or university, we must also 
know how to induce institutions to build the necessary kinds 
of institutional commitments and make the kinds of stmctural 
changes i squired. 

A second purpose of this book, therefore, is demonstrative: 
to provide evidence of the ways institutions have actually used 
assessments of student growth and development to make improve- 
ments in instruction, in curriculum, and in the student learning 
environment. Part 2 of the volume offers evidence about institu- 
tional experience with explicit institutional assessment. In .most 
cases, the research supporting these initial efforts has been both 
unsystematic and technically unsophisticated. In all cases, the 
need for more and better research about the postsecondary 
learning process has been an irnportant product of the utilization 
effort. But the biggest payoffs have occurred because of the effort 
itself. Adopting an explicit program for assessing and improving 
educational outcomes requires an institution to articulate and 
internalize a special kind of collective responsibility. 

Based upon evidence from a number of such institutions, two 
important arguments can be made. First, it is quite possible for in- 
dividual institutions to undertake such efforts without a massive 
infusion of external resources. What campuses require is primarily 
an integration and reorganization of numerous existing student- 
assessment activities. Secondly, the effectiveness of such efforts 
is highly dependent upon their being institution-specific and 
participatory in character. Neither government nor the research 
community can hope to impose solutions— no matter how well 
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informed— if faculty and administrators have not first inter- 
nalized the logic of these solutions through their own evaluations 
and experiences. 

The two purposes of this book, though distinct, remain 
highly related. Institutional efforts to achieve excellence must first 
be informed by an attempt to define excellence from within the 
distinctive environment and perspective of each institution. They 
must secondly b grounded in an attempt to assess the attainment 
of excellence through an explicit, systematic, and participatory 
examination of educational outcomes. In defining what it wants 
its students to be, a college or university essentially defines itself. 
In assessing the degree to which educational goals have actually 
been achieved, a college or university identifies critical areas for 
future action- Self-definition and self-examination constitute the 
first and most important steps toward ongoing sdf-improvement. 
In the absence of these steps, it is impossible to conceive of achiev- 
ing excellence. 
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PART 1 



Structuring Excellence: 
Dimensions of 
Outcomes 



Excellence in higher education, as in anything else, is a 
concept difficult to define yet difficult to disagree with. Lack 
of consensus about the concept is heavily bound up with its lack 
of clarity; every attempt to define its attributes is likely to increase 
the disagreement. Indeed, the notion of excellence seems to con- 
tain an essential contradiction. Its very construction connotes 
exclusivity: to be "excellent" is to be distinctive, superior, and 
■ worthy of emulation. Excellence thus establishes an external stan- 
dard that, by definition, not all will meet. Yet failure to strive for 
excellence— in business, in sport, or in intellectual attainment— is 
considered, especially in American .society, an abdication 
responsibility. Quite aside from the imposed standard, working 
toward excellence is, in itself, a valued process*- a process by 
whjch individuals and institutions attempt to move themselves 
from where they currently are to where they and others would 
like them to be. 

So, is excellence a standard or a process? This conceptual 
contradiction is particularly imbedded in discussions of the 



11 

19 



Peter Ewell 



excellence of postsecondary education and of the institutions that 
provide it. On the one hand, excellence is judged in terms of 
particular sets of standards: available resources, the structure 
of programs and curricula and the intellectual characteristics 
and attainments of faculty and students. (This theme has been 
particularly evident in the process of postsecondary accredita- 
tion./ On the other hand, these sets of standards are taken to 
be indicative of an underlying process. High levels of resources, 
well-structured programs, and an able faculty and student body 
are assumed to be highly correlated with educational growth. 
They are consequently valued largely because of the perceived 
increments of educational growth they are believed to produce. 

So long as most higher-education institutions were essentially 
in the same business, the terms of this contradiction remained 
obscure. The a p p rop ri ate outcomes of higher education were 
relatively few and well agreed upon: a liberal education, with its 
associated value structure, and scholarly attainment within a 
particular discipline Furthermore, the production function asso- 
ciated with these outcomes, though relatively unexamined, was 
comparatively straightforward: a four-year curriculum, built 
around a disciplinary major, offered to e small and carefully 
selected body of recent high-school graduates. 

All of this now has changed. New institutions with new 
missions have been established, and many others have seen their 
missions redefined. Added to this have been vast increases in 
higher-education participation rates over the past three decades. 
Legitimate outcomes of higher education are now claimed in 
many different arenas, and the educational processes used by dif- 
ferent types of institutions to produce these outcomes vary in the 
extreme. Under such circumstances, defining the concept of 
excellence in higher education, either as a standard or as a process 
of educational growth, presents a considerable challenge. To do 
so within the confines of a single traditional dimension of educa- 
tional outcomes is impossible. 
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Given this complexity, the main purpose of part 1 of this 
book is to provide a consistent language for discussing the 
outcomes of postsecondary education. The argument will be that 
institutions achieve excellence in postsecondary education inso- 
far as they produce demonstrable changes along particular 
dimensions of educational outcomes consistent with (1) institu- 
tional objectives, (2) student educational goals, and (3) the 
expressed needs of society and of particular constituencies within 
society. An important distinction in this argument will be be* 
tween the empirical content of the particular changes produced (a 
question of measurement and educational technology) and the 
values placed upon these changes by different interested parties in 
the system (a question of perspective ami preference). 

The implications of this discussion for the accountability of 
higher-education institutions are many. Posing multiple dimen- 
sions of educational outcomes and recognizing the legitimacy of 
many perspectives on the valuation of these outcomes at first may 
seem to deny totally the notion of institutional accountability. 
Clearly, from this point of view, ranking all institutions in terms 
of a single set of evaluative criteria will be impossible. But the 
existence of both multiple outcomes dimensions and multiple 
perspectives on the value of particular outcomes may yield new, 
more useful ways to demonstrate accountability for excellence. 
Such an approach, for example, suggests that all institutions 
should be held accountable (1) for clearly stating what kinds of 
outcomes they are trying to produce, (2) for explicitly assessing 
the degree to which they are attaining these outcomes, ami (3) for 
making appropriate changes to improve the situation where the 
data warrant. Such an approach also implies that groups within 
society with a need for particular educational products haw an 
obligation berth to state that need clearly ami in outcomes terms, 
and to be able to apply a rigorous and consistent set of assessment 
standards. 
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The chapters making up part 1 approach these issues from 
several directions. The first, "Educational Excellence as a Change 
Concept," is intended to clarify some important distinctions 
between the ways we typically talk about educational excellence. 
On the one hand, we otteu view educational excellence in terms 
of absolute levels of attainment; on the other, we often view 
excellence in terms of the degree to which students change as 
a result of the learning experience. And for both of these, we 
tend to confuse the issue of whether attainment or change has 
occurred, which is an empirical question, with how we and others 
feel about it, which is a question of valuation. 

The second chapter, "Some Different Perspectives on Educa- 
tional Outcomes," reviews major elements of the literature on 
college and university outcomes. This literature is vast and 
diverse, and the studies it comprises were intended for many dif- 
ferent purposes. The purpose of the chapter is to integrate these 
various approaches and set the stage for developing a common 
institutional language for describing such outcomes. 

The third chapter, "A Classification of Outcomes Dimen- 
sions," presents the resulting structure along four major dimen- 
sions; knowledge, skills, attitudes/values* and relationships with 
society. An intent of the classification is to indicate dearly the 
differences among different types of educational outcomes, and 
to emphasize that different institutions can legitimately pursue 
excellence along different dimensions. Nevertheless— as the 
fourth chapter argues— various constituencies within society will 
value achievements on each of these dimensions differently. And 
institutions ignore such external valuation at their peril. Indeed, 
much of the content of strategic planning, currently in vogue in 
higher education, consists of matching an array of external valua- 
tions with a defined set of internal capacities and potential 
capacities to produce particular types of outcomes. 

It is a reasonable question to ask, of course, why it is necessary 
to pay attention to the issues at all. If many potential impacts of 
colleges and universities are possible, and if all are legitimate, a 
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language that distinguishes them may seem superfluous. Further- 
m5fe t the extensive literature on college impact would seem to 
provide ample justification for almost any course of action a par- 
ticular institution might wish to pursue. Despite these arguments, 
our current lack of a systematic way of talking about educational 
quality, and about the different directions along which it can be , 
sought, is a growing obstacle to both individual and systemic 
improvement. 

At the systemic level, lack of a good vocabulary on educa- 
tional outcomes has led to considerable confusion in recent public 
debates about the merits of particular postsecondary educational 
systems. Much recent criticism of community colleges, for 
example, has been based upon the relatively low degree produc- 
tion and high attrition rates typical of such institutions. Because 
these institutions have not been defined in outcomes terms, most 
of the right questions in this debate have simply not been asked— 
ruestions regarding the ability of former students to perform in 
an employment skills area, or to benefit from personal self- 
development, or to transfer successfully to a senior institution. 

Indeed, it is important to recognize that most of the major 
developments in the landscape of higher education over the past 
three decades have been additive rather than homogeneous. New 
clienteles have largely been accommodated in new institutions 
with new programs and missions, leaving many established insti- 
tutions essentially unchanged. However, public discussion of 
these changes has, almost exclusively, proceeded as though such 
changes were homogeneous— that all institutions will be affected 
equally by the same demographic trends, shifts in societal man- 
power needs, and so on. Much of this confusion, it may be 
argued, has been due to our own inability to articulate clearly, in 
outcomes terms, what the major differences are. 

At the institutional level, the need for such a language is 
equally apparent. Recent studies of institutional decline have 
emphasized the importance of an institution's ability to state 
clearly its particular mission and the particular types of students it 
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intends to produce. In Chaffee's study of eight small private 
liberal-arts colleges, for example, those institutions that success- 
fully recovered possessed this ability; simultaneously, they 
adapted their programs to changing societal demand. Those that 
did not possess this ability continued to decline despite their 
many efforts to adapt to serve new needs and new clienteles 
(Chaffee 1984). The general principle is apparent: educational 
institutions, like other collective enterprises, cannot hope to 
achieve excellence unless they can effectively communicate to 
others the particular business they are in. Doing so requires 
development of a common language centered not around the 
resources ihey possess or the reputation they have achieved, but 
on what they, in foot, expect and hope to produce. 
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CHAPTER 1 



Educational 
Excellence as a 
Change Concept 



In applying the concept of excellence to education, the same 
term is generally used to refer both to the product of an educa- 
tional experience and to the experience itself. Assessment of these 
two, however, is likely to be radically different. While products 
may be evaluated by reference to a^ static external standard, 
processes can only be judged in terms of the changes that occur 
as a result. Such a distinction between product and process evalu- 
ation is, of course, rarely achieved in practice. Some educational 
products are indeed commonly assessed in process or "value 
added" terms— for example, continuing education or professional 
development. Similarly, some processes we generally assessed in 
terms of standards—for exampfc the standards of curriculum 
structure that are used as part of many professional accreditation 
processes. 

For the most part, however, the evaluation of educational* 
product and educational process are appropriately" distinct 
activities. However, both activities share the same measurement 
technologies. Also, the results of measurement or assessment are 
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presented in terms of the same units. A given test or assessment 
procedure may, for instance, be applied to ascertain a particular 
student^ level of development ar a single point in time. At this 
point, an appropriate question may be asked about the degree to 
which the result of this measurement corresponds to a previously 
established standard. Alternatively, the same test or procedure 
may be administered over time to ascertain the amount of 
development resulting from the educational process. At this 
pqint, an appropriate question may be asked about the dif- 
ferences between* a measurement taken it t|e beginning of the 
process and one taken at its conclusion. Both measurement pro- 
cedures, in order to be meaningful, involve comparisons, but the 
points of reference differ notably. 

More importantly, the .moru'es for comparison are quite dif- 
ferent. The primary reason for comparing the results of a 
measurement procedure to a particular standard is to place a 
value on the product. Many quite different standards may in this 
way be applied to identical outcomes, and the judgments of worth 
entailed will, as a result, be quite different. The primary reason, 
on the other hand, for comparing measurement results at the 
beginning and end of a particular process is to ascertain whether 
a particular change has, in fact, taken place. This is a purely 
empirical question and does not automatically involve valuation 
of the outcome. Furthermore, in contrast to the valuation pro- 
cess, such a procedure should result in only one right answer. 

In the face of these issues, making conceptual sense of the 
notion of excellence in higher education means being clear about y 
two basic distinctions. The first is between assessing the actual 
changes occurring as a result of the educational process (an em- 
pirical measurement question) and making judgments about the 
results of the process (a process of valuation). The second distinc- 
tion is between alternative ways of making evaluative judgments. 
As figure I illustrates, any conceptual treatment of excellence re- 
quires that these distinctions Be approached in the proper order. 
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Empirical Assessment 




What changes have in fact occurred as a 
result of the process? 

f (measurftnent at Tj) 

(similar measurement at T 2 ) 



Valuation 

Valuation of the Educational Product: 

Do the characteristics of the final educational 
product correspond to standard x? 

(measurement at T 2 vs. standard %) 
(maximum possible value of T 2 ) 



Valuation of the Educational Process* 

Is the obtained change of the proper 
magnitude and in the intended direction? 

{difference between Tj ami T 2 vs. expected 
d iffe re n ce) 

(maximum possible difference between 
Tj and T 2 ) 
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Measurement vs. Valuation 



The most important initial distinction is between the pro- 
cesses of measurement and valuation. Primary activities in the 
measurement process include establishing appropriate categories 
and units for measurement, and applying these categories to an 
array of available data about the phenomenon to be measured. 
In principle, neither the choke of categories nor die application 
process is a value-driven process. In practice, of course, we tend to 
choose the categories in terms of an implied valuation scheme; 
this tendency accounts for many of the variations among schemes 
that classify educational outcomes. 

Regardless of its origins, once such a set of outcomes dimen- 
sions or categories is in place, it can be applied to many different 
situations and at many points in time. The resulting outcomes 
profiles can then be compared in many ways. Exiting students 
from different types of institutions or in different types of pro- 
grams, for example, will undoubtedly differ in significant ways. 
Similarly, important differences will be found in the character- 
istics of entering students fironv various backgrounds and with 
various aspirations. Many such comparisons of student profiles 
are possible given the purposes of any empirical investigation. To 
understand the actual effects of a given instructional process, 
however, only one type of comparison mast be accomplished: that 
between student profiles at the beginning of the process ami at 
its conclusion. More importantly, only the comparison between 
before and after will provide the kind of information needed to 
intervene in and improve the educational process. 
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Two Kinds of Valuation 



A quite different set of activities is typical of any valuation 
process. Here the primary object is to assess a given student pro- 
file—however obtained— in terms of a valued external standard. 
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Many types of standards may be appropriate for particular pur- 
poses, and the application of such standards to the same empirical 
profile will yield quite different results. But all such valuations of 
educational outcomes can be grouped essentially into two classes. 

The first set of standards is static and is applied to the student 
on exit. The essential question posed by this type of valuation is 
one of certification— the degree to which the educational pro- 
duct, as currently assessed, is consistent with a desired level of 
attainment .Within this cor ception, educational excellence can be 
defined in several ways. One approach maximizes the proportion 
of graduates meeting the standard on completion of the program. 
An alternative notion of excellence might involve attainment of 
the maximum possible score, in excess of the standard. 

The second set of standards is applied to the actual educa- 
tional gains experienced as a result of instruction. The essential 
question posed by this type of valuation is one of educational 
effectiveness— the degree to which particular gains have been 
achieved consistent with objectives. Here again, excellence can be 
defined in several ways. One approach is the proportion of 
students for whom minimum acceptable gains on particular 
valued dimensions are achieved. An alternative approach defines 
excellence in terms of the maximum possible gains obtained, 
consistent with previously defined instructional objectives. 



A Vforking Definition of Quality in Educational Outcomes 

A major assumption in this chapter s discussion is that higher 
education can only be meaningfully conceptualized as a change 
process. The notion of an "educational outcome" is thus also a 
change concept— the assessed differences in the attributes and 
abilities of a student upon leaving the process as compared with 
three upon entering the process. Instruction changes some 
concrete attributes, and these chinges are, at least in principle, 
measurable. But instruction in itself does nothing to change the 
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value of these attributes—changed or unchanged— to different 

segments of society. 

Within this general framework, a working definition of in- 
structional quality might look something like the following: 

Quality in educational outcomes is attained when the greatest 
net gains in particular student attributes and abilities from 
entry to exit, as assessed along a set of defined dimensions, are 
Accomplished (1) consistent with previously set institutional 
instructional objectives, and (2) consistent with an appropriate 
array of. values placed upon these gains by particular consti- 
tuencies (including students) within society. 

Operationalizing this definition first requires, then, proposal 
of an appropriate general set of dimensions along whkh to array 
concrete educational outcomes. Secondly, typical arrays of values 
placed upon these dimensions by different segments of society 
must be identified and reconciled with particular patterns of con- 
crete outcomes. These activities are the concern of chapters 2 
through 4. 

ft 
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CHAPTER 2 



Some Different 
Perspectives on 
Educational Outcomes 



Attempts to structure the outcomes of postsecondary educa- 
tion have been irany and varied (Lenning 1977b). Some, 
like Astin's fourfold typology, have been developed to support 
particular research efforts (Astin, Panes, and Creager 1967). 
Others, like Harshman's taxonomy of student outcomes (1979), 
Bloom's taxonomy of cognitive objectives (1956), or Chickering's 
"vectors" of student development (1969), have been constructed 
to summarize and inventory the results of a broader field of in- 
quiry. Still others, like Lenning's "outcomes structures for post- 
secondary education," result from attempts to build a common 
language for institutional data exchange (Lenning 1977a). 

Each of these structures was designed for a particular pur- 
pose, and their characteristics reflect their different purposes. 
Each therefore possesses important limitations when applied 
outside its original conceptual environment. Those taxonomies ' 
designed to support and inventory research, for example, often 
classify as much by source of data as by conceptual category. 
"Those designed as standards for communicating institutional 
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data are often alarmingly static and take little account of the com- 
plex causal interactions among outcomes elements. But for the 
most part, it is clear that those who have tried to map the territory 
of postsecondary educational outcomes have been viewing the 
same landscape. 

Our purpose in once again exploring this common concep- 
tual territory is not simply to add to a growing list of overlapping 
classifications and descriptions. Rather, it is to determine some of 
the common elements underlying all such classifications and then 
to use these common elements to propose a set of relatively inde- 
pendent dimensions of excellence in postsecondary educatio n. 
A second purpose is to propose these dimensions not as static 
standards for evaluation but as distinct elements of potential 
student growth and development. 



Cognitive Development 

One way of beginning the process of classifying higher- 
education outcomes is to examine briefly a number of distinct 
traditions of student -development research. The first, and oldest, 
tradition is cognitive, involving the processes and correlates of 
individual student learning. Summative assessments of cognitive 
gain in a college environment go back as far as 1928 to the Penn- 
sylvania General College Tests (Learned and ^food 1938). Experi- 
mental work in the classroom— drawing heavily on learning 
theories developed for elementary and secondary education—has 
at least as long a history (Lenning and Munday 1974; Feldman 
and Newcomb 1969). And indeed, assessment of cognitive devel- 
opment has become, in the past two decades, a major industry 
(see the review in Pace 1979). Commercial testing instruments, 
now a prominent part of the postsecondary landscape, are used in 
functions as diverse as admissions, placement, and professional 
certification. Increasingly, such assessments have become part of 
discussions of institutional accountability as well. 
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The underlying question of all research in the cognitive tradi- 
tion concerns "the process of acquiring concrete knowledge— 
either generd knowledge or knowledge of a particular area of 
study. The unit of analysts for most of this work has been the indi- 
vidual student, although often as a member of a particular popu- 
lation—traditional, minority, or adult. And a major intent Has 
been development erf new instructional technologies— curricular 
structures, learning environments, and presentational methods 
or formats. 



Impacts upon Society 

A second tradition of academic investigation involves the 
impacts of postsecondary instruction upon society as revealed 
through patterns of employment, social mobility, and social 
behavior. Investigations of postgraduate experience for particular 
institutions can be traced back to the mid-193Qs (Eurich and 
Pace 1958; Greenleaf 1939). Attempts to bring these studies 
together to form a more comprehensive picture of the societal 
impact on higher education are, however, much more recent 
(Bo wen 1977)* Indeed, many such investigations me at least 
partially due to mounting public concern with the historical 
"return on investment*' of an increasingly costly public higher- 
education enterprise. 

Research questions in this tradition have been generally more 
diffuse than in the cognitive domain* One set of questions con- 
cerns the congruence between the knowledge and skills provided 
by higher education, ami the needs for such knowledge and skills 
in society. This can be either investigated directly for particular 
industries or interest groups or inferred by the success of grad- 
uates in particular fields finding employment and moving upward 
along a career path. A second set of questions has to do with effec- 
tive social functioning. This concerns the role of higher education 
in providing basic skills such as problem solving, critical thinking, 
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and information gathering important to a broad range of social 
activities (for example, Spaeth and Greeley 1970). A third set has 
to do with the actual changes in social behavior that are assumed 
to follow from such skills. Examples of such changes include 
enhanced political and community participation, increased 
tolerance for cultural diversity, and different approaches to 
marital roles and child rearing (for example, the reviews in Pace 
1974). Within this tradition, the impacts of higher education on 
individuals are not manifested within the individuals, but rather 
in terms of changes in the relationships between individuals and 
particular groups in society. 



The College Experience 

A third tradition of academic investigation explores the 
broader psychological and attitudinal impact of college on those 
who experience it. Rather than being confined to cognitive 
development, such studies examine the development, in the 
course of education, of the "whole person"— attitudes, percep- 
tions, values, and the interactions of these elements (for example, 
Chickering 1969). Investigation within this tradition has not been 
limited to the production of such changes within the college 
environment. It has also covered the persistence of such changes 
after graduation or withdrawal (Trent and Medsker 1968; 
Newcomb, Koenig, Flacks, and Vforwick 1967)* 

Research questions in this tradition generally focus upon the 
kinds of changes in attitudes, perceptions, and behavior that 
actually occur in the course of the educational experience. Such 
questions also emphasize the degree to which such changes can be 
linked to explicit characteristics of the campus or educational 
environment (for example, Astin 1977). The unit of analysis for 
research, as in the cognitive tradition, is the individual student. 
Also as in the cognitive tradition, there is often an explicit focus 
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on action: how can we make appropriate changes in campus learn- 
ing environments designed to produce the kinds of "identities" we 
would most like to see (Astin 1979)? And finally, like that of the 
cognitive tradition, the vast majority of this work concentrates on 
the development of the 18- to 21 -year old student in a structured, 
residential, four-year undergraduate curriculum* 

This third tradition covers from a different perspective some 
of the same question? addressed by the research tradition that 
focuses on societal impact. The development of particular clusters 
or attitudes and values is often identified with the acquisition of 
broad classes of sotial-fuiKtioning skills— for example, Chicker- 
ing's seven "vectors of development" (1969), or more narrowly 
within the college experience itself, Pace s notion of "quality of 
student effort 99 ( 1984). In this sense, a strong concern of this third 
tradition is the production of particular, needed "metaskills," 
presumed to be of value to both the individual and society. 



Occupational Skills 



A final, fourth tradition of investigation is much more recent 
and narrowly focused. It consists of a large number of institution* 
and program-specific studies on the effectiveness of vocational 
and applied skills programs in providing training relevant to the 
needs of particular occupations and industries (see, for example, 
the review of proprietary institutions in Trivett 1974). The uni* of 
analysis for such investigations is generally the program being 
assessed; the primary research question, the degree to which the 
program has provided students with enhanced abilities to per* 
form particular, well-defined tasks in the workplace. A few such 
studies rely upon independent assessments of particular skills 
through an. explicit testing or demonstration process. The ma- 
jority, however, rely upon reports of enhanced effectiveness from 
particular employers, or from students and former students. 
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In marked contrast to the other three traditions, such in- 
vestigations haw rarely taken place in die context of the four-yew 
undergraduate institution. Rathd, they have been conducted 
by and for two-year occupational programs resident in public 
community colleges, corporate training programs, and other 
proprietary settings. 



The Need for Multiple Dimensions 

While the overlap among these four traditions of investiga- 
tion is considerable, each appears to cover a relatively distinct 
dimension of educational impact. Certainly, the case for an inde* 
pendent cognitive dimension seems indisputable, and virtually all 
attempts to structure educational outcomes have broken out this 
element as distinctive. Similarly, there seems a dear case with the 
fourth tradition for treating the acquisition of particular ap- 
plicable skills as an outcome element different from die cognitive . 
The development of attitudes, perceptions, and values addressed 
by the third tradition, on the other hand, is a good deal more 
complex. Proponents of this tradition, indeed, would probably 
subsume cognitive growth and skills development within the 
broader context of developing personal identity. Nevertheless, 
there is certainly something distinctive here, and most taxon- 
omies of outcomes righdy contain an independent cognitive 
dimension* Finally, investigations of the second tradition seem to 
cut across all categories of knowledge, skills, and attitude, paying 
special attention to the explicit ways such categories tend to 
promote or enhance particular relationships between the indi- 
vidual and society. This, too, seems worthy of independent 
consideration. 

The above discussion suggests that a useful way to structure 
the broad dimensions of postsecondary educational outcomes is 
in terms of the diagram presented in figure 2. The essence of this 
scheme consists of three major dimenskms—kn^ledge, sL'Js, 
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FIGURE 2 

Major Dimensions of Postsecfendary Educational Outcomes 

• Knowledge 

• Skills 

• Attitudes/ Values 



• Relationships with 
^Employers 
—Professions 
—Other Interest Groups 
-^Society 



and attitudes — accompanied by the manifestation of these dimen- 
' sions in the subsequent relationships between students and 
particular groups in society. While each dement of this structure 
needs further exploration, several observations seem relevant at 
this point* 

First, the correspondence between this perspective and a 
number of previous ways of classifying outcomes is considerable. 
Knowledge, skills, and attitudes are qutoe similar, for example, to 
"cognitive learning, affective development, and practical com- 
petence," Bowen's classification of the goals of education (1977). 
Astii/s fourfold typology (Asdn, Panos, and Creager 1967) com* 
bines knowledge and skills in a single "cognitive" dimension, but 
his distinction between "psychological" and "behavioral" sources 
of data about outcomes seems similar to the distinction between 
individual ami "relational" manifestations of the three major 
outcomes dimensions* The pr o pose d structure aim covers more 
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succinctly the categories listed under "human characteristics 
outcomes" of the NCHEMS Structure for the Outcomes of Post- 
secondary Education (Llnning 1977a). Chickering's seven vectors 
of student devebpmer/t represent a quite different approach 
( i969); many of Chickering's dimensions, however, can be seen as 
interactions among the proposed elements— particularly those 
dealing with general social functioning. 

More importantly, the categories proposed are not tied to a 
particular institutional perspective. One difficulty with many in- 
ventories is their implicit exclusion of certain types of institutions 
or student bodies— most notably, vocationally oriented institu- 
tions and nontraditional, adult, part-time student bodies- Such 
institutions and students certainly * can bt accommodated in 
almost any classification scheme. In most, however, they appear as 
exceptions to the dominant traditional four-year-undergraduate 
center of gravity. 

Each of these dimensions will be more folly categorized and 
discussed in the following chapter. 
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A Classification of 
Outcomes Dimensions 




n initial task in achieving excellence in educational 
.utcomes is to define the distinct dimensions along which/ 
idt improvements can be attained- Each such dimension can 
conceived of as a 'Vector" of development: a continuum a'ong 
/htch individual student changes can be charted and assessed. e 
Kjferent institutions will put different values on changes accom- 
jplished along each of these dimensions. Liberal- arts colleges, for 
[example, may emphasize attainment of general knowledge and 
the acquisition of basic intellectual skills. Community colleges 
and proprietary institutions may stress mastery of particular 
kinds of specialized knowledge and vocational skills. And many 
institutions with a particular religious or values orientation may 
add to these dimensions student growth in particular areas of atti- 
tudinal or moral development. 

Each institution will, in essence, construct a series of profiles 
on the basis of these dimensions. One such profile, the "educa- 
tional objective'' profile, documents the ideal level,of attainment' 
of the institution's graduates. In contrast, a second profile 
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documents the actual results obtained, based on assessment of 
graduating students. A third * profile—obtained for entering 
students— constitutes a baseline for value-added assessment. 
Explicitly or implicitly, all colleges and univroitift construct 
such profiles in order to plan ami evaluate educational programs. 
A major goal of the classification scheme presented in this chapter 
is to provide a common vocabulary for making such activities as 
explicit and precise as possible. 

The basic structure of die classification scheme js based upon 
the three dimensions of educational outcomes noted at the end of 
the last chapter* Student changes in state are assessed in terms of 
knowledge, skills, and attitudinal/value outcomes. A fourth 
"dimension" is provided by the Iftany behavioral changes occur- 
ring in relationships between students and other Elements of 
society resulting from the educational process- Table 1 sum- 
marizes the classification scheme. Each dimension is discussed 
more fully in the following sections. 



Knowledge Outcomes t 

Knowledge outcomes represent the purely cognitive aspect of 
educational outcomes. They may be distinguished from skills in a 
number of ways. The most straightforward such distinction is 
probably Lenning's (1977a): knowledge consists of familiarity with 
and analysis and comprehension of facts and principles resident 
in a particular discipline or field of study, while skills enhance the 
performance of specific behavioral tasks. 

Student gains with respect to knowledge are usefully treated 
in terms of two broad headings: general knowledge and knowledge 
of specific fields of study. Widely recognized in extant taxonomies, 
this distinction is also being used in at least on* statewide perfor- 
mance evaluation of higher-education institutions (Bogue 1982). 
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TABLE 1 

A Classification of Outcomes Dimensions 

I. Knowledge Outcomes 

A. Genera! Knowledge (Breadth of Knowledge) 

B. Knowledge of Specific Betds (Depth of Knowledge) 

II, Skills Outcomes 

A. General Competence (Social -Functionihg) Skills 

B. Professional /Otoipatfcind Skills 

Ul Attitude/ Value Outcomes 

A- Personal Goals ami Aspirations 

B. General Attitudes* Values, ami Satisfactions 

C. Attitudes towar! Sdf (Development of Identity) 

D. Attitudes toward Others 

IV. Relationships with Society am! with Particular Constituencies 

A. Relationships with Educational Institutions 

B. Relationships with Employers/Industries 

C. Relationship wkh Professions 

D. Relationships with Famtiy/Community/Sodety 



General Knowledge (Breadth of Knowledge) 

/' 

Genera] knowledge consists of familiarity with and analysis 
and comprehension of facts and principles inherent in broad 
areas of study. Such broad areas will generally contain the tradi- 
tional divisions of academic study: the (humanities, fine arts, 
physical sciences and social sciences. Mastery of general 
knowledge is also generally held to include the ability to relate 
concepts drawn from different fields as well as to manipulate such 
concepts effectively, aware of both their analytical potential and 
their conceptual limits. Senior or College Honors examinations 
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in four-year liberal -arts colleges commonly consist of assessments 
of this kind. Standardised attempts to measure this attribute 
include the UAP Area Examinations and the ACT-COMP 
examination (Forrest 1982; Forrest and Steele 1978). 

A list of the component parts of general knowledge might 
include: . * - 

• recall and comprehension of facts in broad areas of study 

• recall and comprehension of theories and terminology in 
broad areas of ftudy 

• recall and comprehension of investigative principles and 
methods in broad areas of study 

• recall and comprehension of the history and development of 
broad areas of study 

• ability to relate/integrate approaches and concepts drawn 
from more than one broad area of study 

Knowledge of Specific Fields (Depth of Knowledge) 

t 

Knowledge of a specific field requires mastery of the accepted 
body of facts, theories, language, and techniques of that particu- 
lar field of study. Such knowledge is most commonly organized 
within established academic disciplines. In addition to mastery of 
the subjecf matter, acquisition of such knowledge will commonly 
include familiarity with app r opri ate standards of inquiry em- 
bedded in a particular disciplinary culture. Assessments of specific 
knowledge are abundant in colleges and universities but are 
generally (and often appropriately) unstandardixed. Examples of 
standardized assessments include the Graduate Record Examina- 
* tions (GRE) Field Examinations and the College Board/ETS 
Advanced Placement Examinations. 

A partial list of the components that reflect knowledge of a 
specific field would include: 
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• recall and comprehension of facts in a particular discipline or 
specialized field 

• recall and comprehension of theories and terminology in a 
particular discipline or specialized field 

• recall and comprehension of investigative principles and 
methods in a particular discipline or specialized field 

• recall and comprehension of the history ami development of 
a particular discipline or specialized field 

• ability to effectively manipulate appropriate concepts, 
theories, and investigative methods to create new knowledge 
in a particular discipline or specialized field 



Skills outcomes consist of the relative abilities of students to 
perform explicit tasks. Some skills, particularly "academic" skills, 
will be strongly correlated with particular clusters of knowledge 
outcomes. Others, such as analyncal/problem-*solving skills, cut 
across specific knowledge areas and represent new areas of appli- 
cation. Still others, such as particular occupational skills, are 
entirely outside the traditional academic arena. 

In general, two kinds of skills outcomes are widely recognized. 
The first consists of skills associated with general competence 
(often associated with basic social functioning). Hie second in- 
volves the particular specialised and applied skills associated with 
individual professions and occupations. 

General Competence (Social-Functioning) Skills 

General competence skills encompass the abilities to use 
knowledge, organize information, define problems, and discover 
and implement solutions to problems. The range of components 
included under this heading is drawn from many parts of previous 
taxonomies of educational outcomes. Most commonly, the traits 
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described as general competence have been created as elements 
of general knowledge. As applied elements, however, tliey are, 
in many ways, quite different from other elements of general 
knowledge. 

It is interesting to note that most public calls for improvement 
of the elementary and secondary educational product refer to this 
cluster of elements rather than to an increase in knowledge per se. 
Up to now, however, explicit, systematic assessments of general 
social-functioning skills have been relatively rare. However, some 
standardized assessment instruments designed to measure 
elements of this dimension currently do exist. Among these, the 
ACT-COMP examination, now in widespread use to evaluate 
general education, actually this dimension much more 1 

effectively than it does general knowledge. 

A partial list of the components of this dimension would 
include: 

• verbal skills, including reading, reacting comprehension, 
writing, and oral communication 

• quantitative skills, including mathematics, statistics, and 
computing 

• leadeirship/o^niiational /human-relations skills 

• analytical skills, including skills of problem definition, 
problem solving, and critical thinking 

• i n vention/ innovation/ creative-thinking skills 

• aesthedc-appredation/creative-expresston skills 

• physical/ motor skills 

Professional/Occupational Skills 

These skills consist of the particular, specialized skills needed 
for effective performance in an identified profession or occupa- 
tion and are most effectively organized in terms of professions and 
occupations. The distinction between professional and occupa* 
tional skills is primarily one of level. Occupational skills will likely 
consist of specific task competencies: for example, the ability to 



36 

44 



CLASSIFICATION 



assemble and disassemble machinery, or the ability to operate 
equipment Professional skills, on the other hand, consist of 
particular aggregations of general social-functioning and 
problem-solving skills appropriate to a given profession. Occupa- 
tions may be described in standard industrial classification terms* 
Professions may be described in terms of recognized, generally 
certifiable career area®, such as engineering, medicine, business, 
teaching, and particular areas of social service. 



Attitude /Value Outcomes 



Attitude/ value outcomes consist of the affective impacts of 
the postsecondary educational experience. Types of affective 
impacts vary from specific, identified satisfactions and dissatis- 
factions with elements of the college experience on the one hand, 
to the development and persistence of complete, structured 
value systems on the other. Indeed, as noted in chapter 2, some 
outcomes classifications are completely made up of dimensions 
within this general area* In such approaches, knowledge and 
skills outcomes are subsumed under more general headings of 
the development of identity and a personal value structure (for 
example, Checkering 1969). 

Because of the wide range of topics covered, any list of 
subheadings within this dimension is bound to be arbitrary. 
Previous classifications appear to clutter around four main types 
of affective outcomes: personal goals and aspirations, general 
value systems, attitudes toward self, and attitudes toward others. 
Each is described briefly below* 

Personal Goals and Aspirations 

Goals and aspirations are defined in terms of levels, patterns, 
and directions of personal interests, desires, drives, and ambi- 
tions. They may be narrow or broad, long-terra or short-term, 
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deeply held or casual. In many cases, these will be career or 
life-style goals. In other, more narrow cases, they may include 
goals for further education and lifelong learning. 

Assessing such goals as a part of the investigation of edu- 
cational outcomes is important for at least two reasons, one 
substantive and the other methodological. The substantive 
reason is that many educational philosophies consciously seek 
to transform personal goals as part of the instructional process. 
The methodological reason for assessing goals is to make better 
sense of particular patterns of student behavior— for instance, the 
decision to withdraw from an instructional program without 
completion of a degree- Measurement options for the assessment 
of student goals are considerable and include a wide range of 
stan d a r di zed psychological- assessment instruments. If goals are 
defined more narrowly in terms of career choice and motive for 
further education, most standardized student-opinion question- 
naires cover this ground adequately. 

A list of the components of this dimension might include: 

• gei ;ral goals and aspirations (life-sty ie, social mobility, 
family goals, personal goals) 

• occupational and career goals 

• educational goals 

• motivation end drive levels for each of the above 

General Attitudes, Values, and Satisfactions 

General attitudes and values can be examined both in terms 
of their extent and content and in terms of their structure. One 
important objective of many educational programs is to expand 
the content and sophistication of individual value structures. 
Another is to make the components of individual value structures 
more internally consistent. Using Lenning's (1977a) basic defini- 
tion, attitudes can be seen as the disposition of an individual to 
react either positively or negatively to something in particular— 
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persons or types of persons, things, or situations, which predis- 
poses an individual to act in particular ways. Defined somewhat 
more broadly, values are strong preferences based upon what is 
seen as desirable, important, and worthy of esteem. Both of these 
elements are embodied in personal belief systems and standards of 
conduct that cut across different aspects of an individual's life. 
Development of such a value structure is held by some as the 
ultimate goal of the undergraduate experience— for example, the 
personalized value structure embodied in Chickering's notion 
of "integrity" (1969). For other kinds of instructional programs, 
however, broad attitudinal development will be seen as entirely 
irrelevant. 

A partial list of the components of this dimension would 
include: 

• beliefs (including religious beliefs), belief systems, value com- 
mitments, and philosophies of life 

• mores, customs, and standards of conduct 

• patterns of feelings and emotions, including particular 
satisfactions and dissatisfactions with individuals, groups, 
institutions, and social situations 

Attitudes toward Self (Development of Identity) 

For some observers, the development of individual identity 
represents the ultimate objective of the undergraduate expe- 
rience. Partially this is a result of confining investigation to a par- 
ticular type of student, the traditional 18- to 21-year-old, who may 
be expected to undergo transformations of personal identity 
simply by virtue of his or her developmental stage. In examining 
adult or nontraditional students, however, it may make sense to 
investigate the degree to which the educational experience has 
reinforced, threatened, or transformed existing self-concepts. 
Discussions of self-concept also overlap with some elements of 
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practical competence, particularly in the areas of leadership and 
organizational skills. Finally, this heading seems an appropriate 
location for the wide range of factors covered by classic psycho- 
logical personality inventories. 

A list of components for this dimension might include: 

• perception of self, general self-concept, self-discovery 

• self-reliance, self-confidence, including adventurousness and 
initiative, autonomy, and independence 

• satisfaction with self, psychological well-being 

• personality /personal coping characteristics, including 
flexibility and adaptability, dogmatism/ authoritarianism , 
tolerance and persistence, and so forth 

Attitudes toward Others 

Like the development of individual identity, development 
of appropriate attitudes toward others is probably worthy of 
separatecreatment as an outcome dimension becauseof its special 
place as a part of the presumed "humanizing" effect of higher 
education. Indeed, the two are generally held to reinforce one 
another: a strong, well-balanced self-concept will aid the develop- 
ment of a tolerant, open, judicious approach to others who may 
have different values and characteristics. 

Possible components of this dimension will undoubtedly 
overlap considerably with general values and belief systems. Such 
components might include: 

• specific perceptions of other individuals/groups in society 

• tolerance for cultural and intellectual diversity, including a 
willingness to accept different points of view 

• general human understanding, including empathy, sensi- 
tivity, and cooperation 
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Relationships with Society and with Particular Constituencies 

Explicit changes in the three major dimensions of outcomes- 
knowledge, skills, and attitudes/values— are presumed to be 
direct results of the educational process. Many of these changes 
may also be assessed directly, through testing instruments or 
other cognitive and psychological-measurement procedures. 
Many, however, will only be indirectly observable through 
induced changes in behavior. Most such behavior will be mani- 
fested in the relationships that individuals create or maintain 
with particular groups or individuals in society. Observation 
and documentation of such relationships thus represent an 
alternative approach to assessing the effects of postsecondary 
educational experience. 

More importantly, particular constituencies in society value 
educational outcomes to the degree that they positively influence 
such relationships. For the employer, for example, production of 
particular knowledge, skill, and atritudinal outcomes is important 
only insofar as job performance is enhanced. Assessment of these 
relationships as independent dimensions is thus additionally 
important because it is these kinds of outcomes that will be first 
and most critically examined by those outside the higher- 
education enterprise. 

At least four key relationships need to be examined, each of 
which deals with a student's interaction with a particular set of 
societal constituencies. 

Relationships with Educational Institutions 

A first set of relationships concerns ongoing participation in 
postsecondary educational experiences. Participation includes 
persistence within a particular curriculum or course of study, 
further programmatic education after graduation or certification, 
and involvement in ad$t education or lifelong learning. Different 
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educational programs, of course, have different intents with 
respect to such participation. Traditional undergraduate liberal- 
arts programs, for example, may include an expectation that a 
student will be both positively oriented toward graduate educa- 
tion and explicitly prepared for a particular program of graduate 
or professional study. Community-college transfer programs 
contain similar expectations for continued participation beyond 
program completion. But many postsecondary programs will con- 
tain no such presumptions and will place no particular value 
upon subsequent educational participation. 

A minimum set of components for this dimension would 
include: 

• individual educational development goals, including 
change, stability, and intensity of such goals 

• patterns of enrollment, placement, and participation 

• patterns of retention, attrition, and program completion 

• patterns of program/institutional change or transfer 

• levels of achievement in subsequent educational experiences 

• patterns of actual student behavior while enrolled (for 
example, quality of student effort) 

• assessed relevance and contribution of past education to 
subsequent educational experiences 

Relationships with Employers/Industries 

Employers and industries represent the largest and most vocal 
consumers of postsecondary educational "products/' Indeed, 
growing numbers are themselves becoming suppliers of postsec- 
ondary instruction. Different kinds of employers will, of course, 
value quite different kinds of educational outcomes, especially in 
particular skills areas. In general, however, the following kinds of 
components seem appropriate: 



42 

£± 50 



• individual employment/career-choice goals, including 
change, stability, and intensity of such goals 

• first job obtained after education/training, including the 
relevance of this job to the education received 

• long-term employment history by occupation and industry, 
including the relevance of positions held to the education 
received 

• income/ earnings history 

• promotion and job performance, including the assessed 
contribution to job performance of the education/training 
received 

• job satisfaction 

v 

Relationships with Professions 

\ Many professions represent organized constituencies distinct 
from particular employers or industries* The primary aims of such 
professional constituencies axe to ensure that standards of com- 
petence and skill are maintained, to certify competence and skill, 
and to faro vide for professional development opportunities for 
members ofthe profession. Examples of such organized, certifying 
professions include such familiar occupations as engineering, 
medicine, teaching, business, and social-service work. Also in- 
cluded under this heading are relationships with such bodies as 
labor unions and organized public-employee groups. 

Particular aspects of this relationship that should be con- 
sidered include: 

• individual professional-development goals, including 
change, stability, and intensity of these goals 

• patterns of professional cerafkation/recognition/award 

• patterns of subsequent professranal-devebpment activities 

• assessed contributions of past education/training to profes- 
sional success 

• professional satisfaction 
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Relationships with Family/Cwnmunity/Sodety 

In addition to the three constituencies discussed above, many 
other explicit societal groups with an interest in postsecondary 
education could be identified. At this point, however, it seems suf- 
ficient to consider all such groups under a single heading. In many 
respects, this group of outcomes represents a behavioral counter- 
part to general attitudes and philosophy of life. Indeed, those 
educational programs that aim particularly at the development of 
individual "identities" generally make significant implicit claims 
about the way such value structures have an impact on social 
function and participation . There is a vast but unsystematic 
literature that treats impacts of this kind, ranging from family 
roles and structure to civic, social, and poUtical' participation . 

Some of the explicit components of this dimension include: 

• family roles, relationships, and child-rearing practices 

• patterns of social affiliation, group membership, and par- 
ticipation 

• patterns of voluntary contribution, including contributions 
of time, money, or other support 

• patterns of citizenship activities/political participation 
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Valuation of 
Particular Outcomes 
Dimensions 



0« 
udining a particular set of outcomes dimensions for post- 
secondary education tells us very little about how these 
attributes should be valued or assessed by diffe re nt constituencies. 
At least three types of constituencies— societal interest groups, 
students, and institutions— will have an interest in such valua- 
tions and assessments. And each will view accountability for 
excellence in a somewhat different way. 

It is important once again to stress that valuation is a complex 
and multidimensional issue. One of the greatest misperceptions 
current among those in colleges and universities is that others in 
society view the outcomes of higher education in terms ifanflar to 
the ones they use. Partly this is because higher education has 
exercised a historic claim on the creation of values. One of the 
primary attributes of the academy in its humanistic rote has 
been to articulate value positions and to constitute a forum for 
examining alternative value structures, regardless of the external 
environment. Indeed, a traditional aim of liberal education has 
been to create values of particular kinds in students. 
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A^single set of value orientations , ho wever broad , is no longer 
a uniform objective of all higher-education institutions. Insti- 
tutions exist for quite different purposes; some espouse quite 
dissimilar value positions and some purport to embrace no value 
position at all. This is one of the major difficulties in explicitly 
adopting the notion of value-added assessment in higher educa- 
tion. The term "value added" is useful in that it calls attention to 
the fact that education is about change, and should be assessed as 
such. But die term "value added" also automatically raises the 
question, whose values? The valuation of particular educational 
outcomes will be quite different for different types of institutions 
and constituents. The actual content of any given value-added 
assessment will, of consequence, be value laden. And such value- 
laden views *%{ society, students, and institutions, as discussed 
below, will result in some quite distinctive views of educational 
outcomes. 



Societal Interest Groupj 

Societal interest groups include employers, professional 
associations, government, and all other groups or constituencies 
with an explicit need for trained or educated individuals. Such 
groups appropriately view the assessment of educational quality 
primarily as a process of certifying the educational product. This 
view is appropriate because of the overriding need of these con- 
stituencies to ensure that the claimed knowledge and skills of 
graduating students are actually present. Indeed, a major concern 
of such constituencies over the past two decades has been the 
steady erosion of knowledge and skill levels of B.A.4evel 
graduates. 

Two trends have resulted from this concern. First, such con- 
stituencies have increased the practice ©(Instituting their own 
assessment programs to ascertain the knowledge and skill levels 
of current ami prospective employees or participants. A recent 
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well-publicized example is the decision of the 1 U.S. Army to no 
longer accept the high -school diploma as evidence of mastery of 
basic skills, and to implement its own4nternal testing program. 
Many professions, of course, have relied on such a practice for 
years, certifying prospective numbers ofrix profession as com- 
petent to practice through an explicit examination procedure. 
The growing number of such programs testifies toa rising concern 
on the part of such external constituencies that tne kinds of out- 
comes assessments practiced by higher-education institutions are 
either inadequate or inappropriate. 

This concern has prompted a second trend: external consti- 
tuents are now themselves providing the requisite training. Many 
corporations/for example, now offer not only in-house training 
in such fields as management, and bask skills development, but 
also specialized training in the requirements of a particular job 
or profession. This results partly from a perception that colleges 
and universities are not meeting this need effectively (Matthews 
and Norgaard 1984). Furthermore, there is some evidence that 
corporations would readily agree that general skills training be 
provided by local colleges and universities if they cculd do so 
more efficiently. 

Different constituencies will, of course, choose different types 
of outcomes as important for certification. Most, however, will 
value outcomes within f''e skills area as the most critical. Most 
employers and professional associations will initially be interested 
in the skills required for effective performance within a particular 
well-specified job situation. In some cases, these will be highly 
specialized and individual skills, such as those associated with the 
operation of a particular piece of equipment or a technical pro- 
cedure. In other cases, the distinctive element is a combincttiotx of 
skills unique to a particular job or profession. In such cases, em- 
ployers and professionals may well insist that actual performance 
in real situations (or simulations of real situations) be assessed in 
order to determine competence, v 
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Employer and professional groups are also interested in 
ensuring that students possess a minimum competence in verbal, 
quantitative, and analytic/pToblem-solving skills. It is in the 
provision of such bask skills that employers, professions, and 
government rely most heavily institutions of higher education, 
and where their concerns about quality have been most vocal. 
Many related concerns center around the degree to which indivi- 
duals have the capacity to learn and releam specialized job skills 
in a rapidly changing environment. A number of industries are, 
in this regard, rediscovering the value of the traditional balanced 
liberal-arts curriculum in providing adaptable and flexible man- 
power. Indeed, some hav* discovered that the kinds of skills 
provided by a general education are often, in the long run, more 
valuable to productivity than those provided by more narrowly 
focused courses of study. 

Who should have the responsibility for conducting such 
assessment is a difficult question. In one sense, the ultimate assess- 
ment is ongoing and behavioral: Do students obtain employment 
and perform effectively in job situations? If the value of postsec- 
ondary education is negligible, the marketplace will soon find it 
out. One option, therefore, is simply to let this process take its 
course. As noted above, a second option, already pr a ctic e d by 
many professions, is to conduct independent assessments of the 
educational product through certification or licensure examina- 
tions. This has the significant advantage of separating the assess- 
ment process from those who provide the education— a principle 
that has considerable proven effectiveness from a learning .stand- 
point. A second advantage of separation is that it allows different 
kinds of assessment to take place for different purposes. Many 
legitimate independent assessments may be simultaneously 
undertaken without the appearance of there being one single 
summative judgment of institutional or program quality. 

Faced with a significant level of external concern about 
student competence, many institutions have begun a process of 
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internal "product assessment" to demonstrate, in a competitive 
manner, the quality pf their graduates. So as the credibility of 
such a process is maintained, this type of testing program may 
have significant advantages over external assessment because the 
detailed results of the process may be directly used in program 
improvement* The second part of this book will contain several 
examples of summati\ r e assessment programs of this kind. 



Students and potential students are a constituency often 
overlooked in discussions of quality. Yet as Pace (1983) has pointed 
out, students are probably the best and most appropriate judges 
of the educational experiences they have undergone. For different 
kinds of students, of course, valuation of the educational experi- 
ence will proceed along quite different outcomes dimensions. 
Some students will seek to gain specific skills appropriate to a par- 
ticular occupation or career. Others will seek general education as 
fulfillment of a much more general perceived need or expectation. 
Many others will not know what they are looking for. 

Despite this variety, the systematic assessment of student 
goals and the comparison of student goals with student outcomes 
have been a strangely neglected activity in higher education. In 
large measure, this has been the case because educators expect 
student goals and aspirations to be uniform. For most institu- 
tions, in fact, degree completion is assumed to be a primary 
reason for attendance. 

For many types of students, however, the actual completioh 
of a program may be only a small part of an educational agenda. 
Much attrition occurs because students perceive that program 
completion at a given institution is not fulfilling their broader 
educational goals, indeed, what students are actually looking for 
may only be discovered at many institutions when they drop out 



Students 



49 




Peter Eweil 



N 



without having found it. But a great deal erf attrition also occurs 
because broader educational goals have been met, and explicit 
program completion was not one of than. 

This is a situation increasingly recognized by community 
colleges and other vocationally oriented institutions. Students 
may attend such institutions purely to master a particular 
employment-related skill or to attain a given level of basic-skills 
proficiency. Consequently, they are not interested in the many 
other potential benefits of completing a degree program and will 
withdraw from such a program when they have obtained what 
they were seeking. Similarly, many two-year transfer students 
f will decide to move to a senior institution the moment they feel 
that they can successfully move. They do so regardless of their 
status in the completion of an associate degree at the institution 
in which they were originally enrolled. Such students— often 
counted as program dropouts— have actually successfully at- 
tained their own valued outcomes with respect to a particular 
educational institution. 

As a result, many such institutions are beginning to recognize 
the vast disparities in the goals of enrolled students. Often this 
will entail systematic assessment and classification of different stu- 
dent "prototypes" of the kind being investigated by community 
colleges in California (Sheldon 1981). In other cases, it will simply 
entail recognition of the disparity of student goals and will temper 
student success and retention analyses as a result (VtWleri 1981). 

Traditional four-year institutions have historically been less 
subject to great disparities in student goals— partly because their 
admissions processes select students most clearly matching their 
own institutional environment. Even within such institutions, 
however, student goals, as revealed by particular behavior pat- 
terns, can come as a surprise to faculty and administrators. Astin, 
for example, has recently documented a broad shift in student 
values and aspirations as revealed through survey responses and 
choice of academic program (Astin 1984c), In addition, a number 
of institutions, after investigating patterns of student changes of 
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major, are discovering that'these arc less related to subject matter 
than to social or career aspiration. The choice of program is based 
upon an anticipated income and life-style, not an inherent in- 
terest in the subject tc be mastered* (Frustrated biology majors, 
for example, enter pre-law programs rather than pursuing 
another science, largely because they originally aspired to the life- 
style and income associated with medicine, not necessarily to a 
medical career.) 

Assessments of educational quality, of course, should not rest 
solely on "consumer satisfaction." Indeed, an avowed purpose of 
many institutions of higher education is to change student percep- 
tions and aspirations and, consequently, to shap* the patterns of 
student valuation. And there is ample institutional evidence that 
such value shifts do occur in the course of a four-year liberal-arts 
curriculum. 

But it seems equally clear that any institution that does not 
pay systematic attention to the match between student aspira- 
tions and actual educational outcomes is doing both its students 
and society a serious disservice. Students will, in fact, differ in 
what they wish to accomplish through higher education; we must 
not assume that the act of enrollment itself automatically entails a 
decision on the part of a given student to accept the values of a 
particular institution as his or her own. And these differences will 
have considerable impact on the ultimate outcomes. 



Institutions 



Institutions have the greatest responsibility for assessing 
and valuing educational outcomes. Different institutions will, of 
course, have vastly different conceptions of the types of outcomes 
they intend to produce. Despite these differences, all have a 
responsibility to articulate such outcomes, to examine their 
success in attaining these outcomes, and to make changes in 
programs and services to improve their effectiveness. Several 
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examples of the way particular institutions have discharged this 
responsibility will be presented in the following chapters. 

Regardless or the institutional value system imposed, there 
are at least two distinct ways of conducting such assessments. The 
first is the most straightforward— evaluation of the effectiveness 
or quality of a particular institution or pr ogr am should consider 
the absolute increments of gain achieved on a set of identified, 
valued outcomes dimensions. This requires that particular valued 
outcomes be identified in advance and that an explicit mechanism 
for assessing gain be put into place. Results of assessment may 
then be used to identify programmatic weaknesses and make 
needed changes. 

A second way of looking at assessment is to look not simply at 
absolute gains, but at patterns or profiles of development across 
dimensions. For many institutions, the shape of the outcomes 
profile— that is, which dimensions are high relative to others— is 
ut least as important as absolute gain on any given dimension. 
This may be especially true of institutions with a particular values 
orientation based upon religion or educational philosophy. 
Furthermore, because precise measur e me n t of many outcomes 
dimensions is impossible, evaluations of student profiles on entry 
and exit may be of greater ultimate value to institutions than 
attempts to measure absolute gain. In practice, most institutions 
will probably want to use both approaches; informally, in fact, 
many already do. 

Regardless of the values placed on higher education by 
different segments of society, institutions have a substantial 
independent responsibility to implement explicit, systematic, 
outcomes -oriented assessment programs to improve the quality 
and to maintain the credibility of their educational products. 
And, as the examples in the chapters of part 2 will show, such 
explicit assessment programs are far more viable, less costly, and 
less disruptive than many may think. 
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Achieving Excellence: 

The Attributes of 
Successful Institutional- 
Assessment Programs 



Being able to articulate educational goals effectively does 
not amount to achieving them. Institutional improvement 
will require taking a wick range of actions. Some actions will con- 
centrate on the curriculum, some on faculty and instructional- 
resource development, and some on the configuration and 
content of ihe learning environment. THe particular mix and 
substance of actions directed toward improvement will, of course, 
depend upon individual institutional circumstances. Each such 
institutional -improvement strategy will, however, be successful 
only to the degree that it is planned, coordinated, and evaluated 
in the context of good information. 

The objective of part 2 of this book is to illustrate some basic 
design principles for gathering and using such information. These 
principles are based upon a growing body of institutional experi- 
ence. To begin the discussion, chapter 5 provides examples of 
three particularly successful institutional-assessment programs. 
Each is located in a distinct setting, and each approaches the task 
of assessment in a somewhat different way. Chapter 6 discusses 
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some of the most common objections to establishing such a 
program at a college or university campus— among them, faculty 
resistance, excessive cost, and Ukk of administrative incentive. 
Finally, chapter 7 outlines some important themes common tc all 
successful efforts and illustrates them with examples drawn from 
actual institutional practice. 

To anticipate, all efforts to establish institutional self- 
assessment and improvement programs share four basic 
requirements: 

1- Such efforts must be founded upon attempt* to articulate 
the specific educational goals of the institution as explicitly and 
as comprehensively as possible. In short, each institution should 
regularly and systematically address the question cf how it will 
affect its students and along which kinds of outcomes directions 
it intends and expects positive changes. Most current statements 
of institutional mission— particularly those typical of public 
institutions— imply an intent to be everything to everyone. Such 
statements are of limited value for setting institutional direction. 
Moreover, considerable evidence suggests that institutions 
seriously attempting to implement such strategies, particularly in 
hard times, are ineffective as a result (Chaffee 1984). Needless to " 
say, the process of articulating instructional goals must also be 
highly participatory. Greatly dependent upon faculty involve- 
ment and commitment, this process may proceed most effectively 
at the instructional unit or department level 

2. Such efforts must be founded i'|H)n visible, explicit, 
institution-specific information. Such information can be of 
many kinds drawn from many sources, but it must be collected 
systematically\nd organized in terms of specific outcomes cate- 
gories identified in the initial phase of assessment. It is important 
to stress that while information of this kind should fa£as accurate 
as is feasible, standards of accuracy are less important than are 
standaids of relevance* Most data that enter such an assessment 
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process will be suggestive rather than determinative. Indeed, a 
successful data-gathering effort designed to support institutional 
assessment depends on its use of multiple criteria and measure- 
ment devices. There is a growing body of evidence to suggest that 
the explicit presence of concrete information in an assessment 
situation may, in itself, have as much to do With generating 
improvement as its empirical content or its direct implications 
(Braskamp I9f&). In addition to informing policy, explicit data 
collection serves as a symbol of commitment and a focus of atten- 
tion (Feldman and March 1981, Ewell and Chaffee 1984). Critical 
to this process are institutionwide structures that require both 
extensive and formal participation across units and departments 
in discussions of the implications of assessment data. 

3. Such efforts should be equally founded upon a set of 
administrative incentives to reward those in the institution 
who -are willing to undertake Informations-based qualitative 
improvements in programs and services. Such strategies will 
take many forms, but all should include mechanisms for effective- 
ly disseminating student-assessment information, for encouraging 
individual academic departments and support units to make 
changes as a result of this information, and for using ongoing 
assessment information to monitor the effectiveness of any 
changes made. Evidence that implementation of such strategies is 
feasible has grown considerably with completion of two national 
projects on the institutional use of information on student out- 
comes (Ewcll 1984; Kemerer, Baldridge, and Green 1982), and 
with a number of exemplary individual institutional programs. 
Successful programs of this kind take substantial care to 
disseminate assessment information to the campus community in 
an accessible, nontechnical, and issue-oriented manner. On most 
campuses, such information, if disseminated at all, is com- 
municated in the form of technical reports that are as daunting 
as they are irrelevant to the decisions that faculty and practicing 
administrators have to make. 
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A second characteristic of these exemplary efforts is that 
assessment information, once disseminated, is used to structure a 
campus dialogue. In many cases, this dialogue is formal and writ- 
ten, consisting of unit responses to questions about unit-specific 
assessment findings in the context of a formal strategic planning, 
budget planning, or program-review prot ss. In other cases, the 
dialogue is less formal and highly participatory, consisting of 
faculty /staff retreats or committee discussions centered around a 
particular issue of student success revealed by trie assessment. In 
both cases, a key to success has been the role of assessment infor- 
mation to raise questions rather than to make judgments. Like all 
incentive systems, those that rely upon rewarding success are 
more generally effective in the long term than those that rely on 
imposing negative sanctions for poor performance. 

4. Such efforts should be designed to include appropriate 
participation on the part of faculty, adminis tration, and 
students. The results of assessment should be used in such a 
manner that further discussion Is encouraged and self- 
sustaining. The ultimate objective of institutional self-assessment 
programs is to encourage active discussion of educational issues, 
policies, and content. As in any other educational process, the 
intent is to focus faculty and administrative attention on 
evaluating, restructuring, and delivering the curriculum and 
institutional learning environment. In most institutions, there 
are at present few opportunities for promoting this kind of discus- 
sion and involvement. Most administrators are rightly concerned 
with day-to-day problems. Faculty members are similarly con- 
cerned with research and service activities. Such activities hardly 
constitute distractions. Nevertheless, most institutions offer few 
opportunities cr forums that allow professional time to be 
allocated to the examination of teaching and learning. 
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Combining these four basic requirements for an explicit, 
comprehensive, information-based assessment structure is not a 
simple task* Furthermore, each campus must approach the task 
on its own terms and within the limits and opportunities pro- 
vided by its own context. Certainly, successful efforts share a 
number of characteristics, but it is important to recognize that 
perhaps the most salient characteristic of successful efforts has 
been consistency of implementation throughout the campus. 
This lends an organic quality to the undertaking. All successful 
efforts involve, as a prerequisite, a dear understanding of educa- 
tional objectives. Each campus engaged in such an effort has 
made a conscious choice as to which of many possible types of 
educational outcomes it most intends to achieve. Each, therefore, 
has a clear idea of what "educational excellence" means within its 
own particular context. And it is important to recall that many 
such definitions of excellence are possible and legitimate. 
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Three Examples 'of 
Institutional 
Self-Assessment 



Because of the many differences among post secondary insti- 
tutions, development of an institutional self-assessment 
process will be far from uniform. Different institutions have 
developed different processes for different purposes* depending 
both upon their individual structures am] upon what kinds of 
educational outcomes they would like their students to acquire* 
The three briefcases presented in this chapter reflect this diversity. 
They also emphasize that creation of a campus wide assessment 
structure is an organic exercise, reliant as much upon the relation- 
ships among its parts as upon the particular tools, techniques, 
and strategies that it employs. 

At a later point we will discuss some of the common elements 
among all such undertakings. For the moment, however, it is 
important to consider such assessment efforts independently. 
Described below are three individual examples of institutions that 
have used self-assessment to address particular sets of problems: 
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Alvemo College— Assessment and Institutional Culture 

Assessment at Alvemo College is strongly conditioned by 
the institution's distinctive history and characteristics. Located in 
Milwaukee, Alvemo is a liberal-arts women's college with 1,500 ' 
degree students and a strong orientation toward preparing women . 
for professional careers. In 1973, Alvemo adopted a new outcome*- 
oriented curriculum structure centered around eight Jbask skills 
dimensions: communications, analysis, problem solving, values 
in decisionmaking, social infraction, taking responsibility for the 
environment, involvement in the contemporary world, and 
aesthetic response. These basic abilities are denned in six increas- 
ingly complex levels that are infused throughout the curriculum, 
which remains configured in terms of traditional academic and 
professional subjects. In designing and delivering particular 
courses, faculty are expected to select levels of the abilities to 
teach toward, and assess these abilities within the structure of the 
discipline. 

With this "outcomes matrix" in place, faculty at Alvemo 
turned their attention to how such outcomes might be 
systematically assessed. Their first task was descriptive: to create 
criteria or descriptive statements that present a picture of the 
ability to be assessed. Their second task was evaluative: to use a 
variety of assessment techniques to match individual student per' 
form?nces with the descriptions embedded in the criteria at each 
ability level. In accomplishing the latter task, Alvemo explores 
many techniques. Among these are the use of external assessors 
drawn from the Milwaukee business and professional community, 
and the assessment of student performance in specific task or 
decisionmaking situations. Information derived from those 
assessments is used to evaluate individual student performance. 
Results of assessments are provided to each student and are used 
to help the student improve her performance. 

A key element of assessment is an explicit place for such 
activities— an Assessment Center, which administers these 
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assessments external to courses. Another element is the Assess- 
ment Council, q faculty body that monitors assessment across the 
* v college and provides technical assistance to faculty. In addition, the 
Office,of Research and Evaluation, funded by the college, system- 
atically evaluates the curriculum as a whole, researches student 
development and the teaching-learning process, and provides a 
way for faculty to investigate their questions about student out- 
comes. Begun with external support from the National Institute 
of Education (NIE) and other sources, the Office of Research and 
Evaluation recently completed a seven-year longitudinal study of 
the impact of the Alverno curriculum as a whole on students, 
both during and after college. 

This investigation was both comprehensive and exhaustive. 
The office studied over 750 four-year and 60 two-year students, 
using intensive interviews, performance assessments, and sixteen 
different assessment instruments. At tjie same time, professionals 
in several fields were surveyed to determine the kinds of skills and 
characteristics typical cf the successful practitioner. Throughout 
the process, primary emphasis was placed upon validation through 
multiple measurement. Most attributes were measured in several 
different ways and at different points in time, using both locally 
developed and commercially available assessment instruments. 
Examples of the latter include Learning Style Inventories by Kolb 
and several exercises developed b v McBer and Company, the 
Watson-Glaser Critical Thinking Appraisal, and measures of 
moral, ego, and intellectual development as described by Kohl- 
berg, Loevinger, Piaget, and Perry. 

Results also help faculty determine if particular elements of 
the curriculum are effective in meeting previously defined educa- 
tional objectives. Indeed, a number of faculty have expanded 
upon the central data-collection effort by engaging in focal 
assessments of their own seminars or programs (Mentkowski and 
Doherty 1984, p. 157). Finally, results of the evaluations have 
been used to validate the curriculum as a who!e. In this regard, 
particular attention has been paid to longitudinal studies of how 
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students develop as they encounter die curriculum, and the per- 
silence of general competencies after graduation. For the most 
part, research showed that values ami skills outcomes do persist 
and are recognized as important by students. 

A primary aspect of assessment at Alverao is that it is firmly 
grounded in a theory of student development. Multiple studies 
are undertaken not simply to obtain "triangulation" of results, 
but also to understand the way students change in the course of 
their enrollment. This focus on development has many important 
implications. Not the least of these is that both faculty and 
students see such a focus as recognition of their importance as 
individual teachers and learners. As a result, they are willing to 
internalize the assessment process fully. 

in sum, Alverno presents us with a case of an institution 
uniquely committed to integrating instruction, assessment, re- 
search, and evaluation* Consequently, theirs is a process that 
allows both educational improvement and un opportunity to 
demonstrate to the outside world that education does indeed 
make a difference. As Mentkowski and Dbherty put it, "(Ttiis 
model} enables faculty to measure things they really care to 
change, instead of measuring outcomes for^ which they are held 

accountable but that are not their own goals That puts the 

faculty in the position of being able to join with the administra- 
tion and with the institution as a whole in explaining to the rest of 
society what education do**-" (1984, p. 160). 



Northeast Missouri State University— 
Linking Outcomes with Resources 

In contrast to Alverno College, Northeast Missouri State 
University (NMSU) presents a mote generally familiar picture. 
Located in the small town of Kirksville, NMSU is a public re- 
gional comprehensive university with a largely residential student 
body of approximately 7,000. Traditionally organized, NMSU 
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offers an array of programs that most institutions of it* site and 
type would recognize. What is less familiar is that for the past five 
years NMSU has engaged in a comprehensive assessment of 
"value added" in its student body, using readily available stan- 
dardized testing instruments (McClain 1984). Furthermore, the 
institution has used the results of this process explicitly, both to 
make resource-alkxxnon decisions on campus and to support «ts 
requests fi r program-improvement dollars from state funding 
authorities. 

Assessment at NMSU involves three linked activities. The 
first of these consists of a value-added assessment of student gains 
in general education between the freshman and junior years. In 
order to assess value added, students are administered the ACT 
Assessment at the beginning of their freshman year (most, in fact, 
enter having taken the ACT) and again at the end of their 
sophomore year. More recently, half the student body has been 
given the ACT-COMP (College Outcome Measures Project) 
instrument to supplement the value-added portion of the assess- 
' ment. Difference scores are calculated on the b$sis of freshman 
and sophomore performance, then reported td the academic 
community, j 

The second activity in the NMSU assessment program is a 
standardized test of achievement in the major field, administered 
at the completion of each student's program. Where applicable, 
all students complete the relevant ORE field examination, or a 
preprofessional or certification instrument where available. 
Scores for NMSU students are then compared with national 
norms, and where possible, subscores are used tq evaluate par- 
ticular portions of the curriculum. \ 

The third activity in NMSU's assessment project involves 
surveying student opinion using several standardized attitudinal 
instruments administered at different points in the student's 
career. Student opinions are gathered by means of questionnaires 
for entering, continuing, and graduating students, ai^d alumni 
follow-up studies. As at Alverno, students are informed about the 
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assessment process from the beginning and are encouraged to see 
the process as symbolic of the institution's commitment to 
evaluate and improve instruction. 

In contrast to Alverno, however, assessment at NMSU is a 
relatively low-overhead operation* Coordination of die process is 
centered in the office of the Dean of Instruction, and assessment 
data are distributed to academic division heads ami student- 
service personnel for local interpretation and use. Because the 
process relies heavily upon existing standardized instruments for 
which computer scoring is available, it is neither expensive nor 
difficult to administer. 

Results of the assessment process have been used at NMSU in 
many ways. Several individual curricula have been strengthened 
as a result of senior achievement test results. Score gains on the 
value-added assessment have pointed to the need to increase 
writing requirements across the curriculum* In a number of cases, 
changes in the curriculum have resulted in marked improvement 
in the competitive performances of NMSU students on national 
normed achievement tests. Most strikingly, remits of the assess* 
ment process have substantiated NMSUs appropriations re- 
quests for the past three year*. Requests for additional state 
dollars have been explicitly linked with proposed improvements 
• r . student performance in identified areas; appropriate targets for 
such improvements have been identified for several future years. 
This approach to funding represents a radical departure from the 
traditional enrollment-driven model, and it is one that many 
public institutions are carefully watching. 

Compared to the Alverno model, assessment at NMSU is a 
limited activity. Rather than developing an assessment process 
from the ground up, NMSU has chosen instead to commit limited 
resources in an attempt to match existing assessment instruments 
with a more tradiripnal set of program offerings. While the scope 
of the effort has been limited, the impact on the campus has been 
considerable. As at Alverno, the largest impact has been the way 
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in which the assessment process has focused faculty and adminis- 
trative attention on student learning and development. And as at 
Alverno, this attention has many implications for the credibility 
of the educational product. As President McQain (1983, p. 6) 
writes in a description of assessment at NMSU, "The model 
NMSU has implemented is one that can be used by any institu- 
tion of higher education . . . . Through the use of this value-added 
system of assessment, the University aims to maximize its human 
and technological resources for the education of its students. It 
aims to be sure it is providing quality education which assures its 
students of competency in the marketplace upon graduation." 



The University of Tennessee, Knoxville— 
Performance Funding and Institutional Response 

As a large, complex public research institution, the Univer- 
sity of Tennessee, Knoxville (pTK) requires an institutional 
assessment vastly different frorft these of Alverno College and 
Northeast Missouri State University (NMSU). Although teach' 
ing is a strong priority at UTK, it is by no means the sole focus of 
institutional activity. Nor can it be* said that UTK has r single 
campus culture or identity. Like most large research institutions, 
UTK consists of many such cultures existing simultaneously in 
the university's schools and departments. 

The roots of assessment at UTK lie in a unique approach to 
budgeting in statewide public higher education. In 1979, the Ten- 
nessee Higher Education Commission (THEC) initiated a pro- 
gram termed "performance funding," setting aside a designated 
percentage of the states annual higher-education budget (initially 
2 percent, no# 5 percent of the total educational and general 
budget) for distribution to individual institutions on the basis of 
identified criteria for institutional performance (Bogue and 
Brown 1982). Currently, funds are awarded according to five such 
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criteria: (1) the percentage of programs eligible for accreditation at 
the institutions that are accredited, (2) student achievement in 
general education (value-added), (3) student performance in the 
major field, (4) student satisfaction with the educational experi- 
ence, and (5) the existence of an established {dan for using the 
results of assessment to improve educational programs. The 
incentives for assessment provided to institutions b) his program 
are considerable. Much, however, depends upon how each 
campus elects to respond. 

For UTK, an opportunity for helping develop a response came 
with the institutions participation in the NCHEM S/Kellogg 
Student Outcomes Project, a three-year, multi-institutional effort 
to demonstrate the use of student -outcomes information in 
program planning ami decisionmaking (Banta 1984). The project 
provided UTK with a limited amount of funding to develop cam- 
puswide assessments of learning outcomes consistent with the 
THEC criteria- To begin the project on campus, three task forces 
composed of faculty and administrators conducted extensive 
reviews of assessment methodologies in general education, 
achievement in the major field, and student satisfaction. Recom- 
mendations of these task forces— transmitted to the chancellor 
and to chief academic administrators in each of the university's 
sixteen colleges and ^chools— served as the basis for constructing 
a comprehensive assessment process for the university. 

As at NMSU; a first component of this assessment process in- 
volves administering the ACTCOMP examination to freshmen 
and to seniors to asses? the "vahle added" of the collegiate expe- 
rience in general education. By analyzing relationships between 
COMP subscores and many other factors (for example, patterns 
of student course work), the university has a good basis for identi- 
fying elements of the curriculum that contribute most to general 
education. Results of this analysis are used by a campus wide coor- 
dinating committee on general education and by counterpart 
committees in each college as they review and revise particular, 
curricula. \ 
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A second component of the UTK assessment effort involved 
initiating pilot projects in fourteen departments to experiment 
with new methods of measuring outcomes in the major field. 
Seven departments used standardized tests or locally developed 
instruments to measure cognitive learning gains; seven others 
surveyed currently enrolled students, program graduates, * or 
employers of program graduates to assemble evaluative data. As 
at Alverno and NMSU, the process of selecting or designing a 
comprehensive examination in the discipline was particularly 
important in bringing department faculty together explicitly to 
consider curriculum goals and objectives. The success of these 
pilot efforts has resulted in a directive from the provost's office 
that comprehensive testing of graduates be considered by the 
faculty of each of the university's 109 degree programs, 

A third component of the assessment project at UTK in- 
volved faculty -designed instruments for assessing student and 
graduate satisfaction. The instruments and methodologies for 
these assessments were designed by UTK faculty in sociology and 
political science, with considerable consultation with the campus 
community. Therefore, administrators, department heads, and 
faculty were given a vested interest in the results. In several 
instances, the general university survey has prompted an interest 
at the department level in "conducting more detailed follow-up 
inquiries. 

Because of the size and complexity of UTK, central direction 
of Teaming assessment has been critical. A key aspect of the assess- 
ment effort is the role of the project coordinator— a full-time facul- 
ty member in educational measurement and psychology, whose 
office is in the university's Learning Research Center. In addition 
to assembling and documenting the results of various assessment 
studies, the- coordinator also serves as a technical consultant to 
schools and departments wishing to coqstruct and administer 
their own assessments. A particularly important responsibility of 
the coordinator is the construction of individual college or 
department reports that succinctly summarize results, by unit, in 
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all three areas of assessment (general education, achievement in 
the major field, and student satisfaction). These reports are 
followed up with in-person consultation between the coordinator 
and the unit head (often including selected or interested members 
of the unit's faculty) to develop the specific implications of assess- 
ment information. These consultations have resulted in 
numerous changes in departmental policies, curricula, and 
organization. 

In contrast to Alverno and NMSU, assessment at UTK has 
been both more decentralized and more reactive. As a large, com- 
plex research institution with a strong departmental structure, 
UTK's assessment appropriately places considerable response 
' bility upon individual academic units. Individual units have been 
given a great deal of latitude in developing and using their own 
assessment instruments and procedures. And indeed, those 
designing their own procedures, rather than simply adopting 
existing measures, also have been the most successful in usinj. the 
results. But the role of central administration in providing incen- 
tives and resources to- undertake assessment has been equally 
important. The use of outcomes information in the university's 
program-review and budgeting processes, as well as the willing- 
ness of central administration to support the technical assistance 
"role of the project coordinator, has been particularly important in 
securing a high level of departmental response to assessment. 

Not least important to the success of assessment at UTK, of 
course, has been the THEC performance-funding initiative. 
Without the external financial incentives provided by perfor- 
mance funding, it is probable that an assessment project of this 
type and magnitude could never have been undertaken at a large 
public research university. Furthermore, had the institution 
chosen to respond defensively rather than creatively to the 
challenge of performance funding, such results would have been 
.y*i squally impossible. In sum, the UTK case is a superb example of 
how external agencies and central administrations can work 
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together r o create a structure of incentives and opportunities for 
individual departmental faculties tc examine critically and, sub- 
sequently, to improve curriculum and instructional practice. 

Each of these three institutions has approached the task of 
assessment in its own way, consistent with its unique goals, its 
particular institutional philosophy, and its available resources. 
All, however, have designed and implemented programs based on 
explicit information about educational outcomes. These pro- 
grams then use such information in concrete ways to stimulate 
program improvement* In building their programs, all have also 
had to find answers to many objections—internal and external— 
to establishing measurement-based assessment mechanisms. 
Some of the most common such objections, and the ways they 
can be met, are considered in the following chapter 
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Some Common 
Objections to 
Institutional 
Self-Assessment 



The three cases discussed in the previous chapter are in many 
respects atypical of most higher-education institutions, both 
in the extent and the quality of their self-assessment efforts. As 
such, it is easy to dismiss them as "outliers" in the more «w*neral 
distribution* of institutional types. Alverno is a womer. allege 
with an explicitly nontradirional curriculum. Its self-assessment 
efforts have been particularly bound up both with the availability 
of external funds to support the effort and with the uniquely 
innovative qualities and commitments of the institution's basic 
identity. Northeast Missouri-State University (NMSU) is, in many 
ways, quite traditional. But it possesses an institutional leadership 
unusually committed to self-assessment. NMSU is also character- 
ized by an administratively lean management style, which fosters 
a good deal of initiative and innovation at the unit level. The 
University of Tennessee, Knoxville (UTK) has experienced the 
mixed advantages of being the major research institution in the 
first state to experiment with funding higher education on perfor- 
mance rather than enrollment. It could well be argued by those 
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reluctant to undertake such programs that institutions such as 
these are, for various reasons, special cases and that their experi- 
ences are consequently of little relevance to most postsecondary 
institutions. 

Indeed, the common response to most innovations in higher 
education is, That's all very well, but it won't work here." The 
vast majority of objections, however, tend to fall into one of three 
basic categories. Some illustrations erf how each type has com- 
monly been met are certainly in order. 



Objections Based on Faculty Resistance 

Many of the most vehement objections to the systematic as- 
sessment of institutional impact will come from faculty Generally, 
these objections are of two quite different kinds, although the 
rhetoric of objection will often combine the two. A first reason 
for resistance is a fear on the part of faculty that they will be 
negatively evaluated. A second basis is more philosophical: a 
conviction that the outcomes of what they do in the classroom 
are inherently unmeasurable by anyone but the faculty. 

Turning to the first source of resistance, faculty often confuse 
assessments of program effectiveness with course evaluation— a 
process that, on many campuses, is notoriously inaccurate or 
biased. In many cases, these two quite different processes have 
not been well distinguished. A key element in securing faculty 
cooperation from the outset, therefore, is to ensure that the focus 
of assessment is placed quite clearly on the curriculum, and that 
faculty are themselves fully involved in the process of designing 
key aspects of the assessmerr. 

One of the most important aspects of the assessment process 
at UTK, for example, is its focus on curriculum in the individual 
department. In the political science and geography departments, 
patterns of student-assessment information led faculty to question 
the effectiveness of the department's introductory course. In both 
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cases, the introductory course was shown to be too unfocused* 
and in need of redirection. Subsequently, an effort was under- 
taken to improve these courses. In one of these departmentyhe 
investigation did involve looking at the relationship between 
individual instructors' and assessment results. However, these 
data were handled extremely carefully by the department chair. 
Course-specific results became the basis of conversations between 
the chair and individual faculty members, but these conversations * 
were not included as part of a formal performance evaluation. 

Faculty resistance based upon fear of negative evaluation is 
. often heavily bound up with more basic objections to measure- 
ment of any kind. Many faculty are simply philosophically op- 
posed to explicit outcomes measurement. T^ey feel that it is 
inherently misleading, oversimplifying, or inaccurate. Moreover, 
many faculty believe that assessments designed to tap general 
attributes do not adequately reflect the specific emphases that • 
they feel are present in their classrooms. 

Two ways of meeting these objections are well illustrated by 
the three casesjpresented earlier. The first approach is to recftgnize 
publicly the inadequacy of any single outcome measure or indi- 
cator and to collect as many measures of program effectiveness as 
possible. This theme is strongly apparent both in Alvemo's 
triangulated approach, in UTK's construction of department and 
school profiles. that merge the results of many data-collection 
efforts, and in NMSU's use of information on both cognitive per- 
formance and student satisfaction. 

The efficacy of this approach may be illustrated by many 
other institutional examples. When the State University of New 
York at Albanf (SUNY Albany) distributed department-specific 
results of an alumni survey as part of the institution's annual 
planning/budgeting cycle, both quantitative and verbal responses 
were provided. Interviews with department heads later indicated 
that the presence of both kinds of information built much greater 
confidence in the accuracy of assessment information. At North 
Carolina State University (NCSU), the assessment process 
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brought together data on black student recruitment, retention, 
academic achievement, campus perceptions, and student behavior. 
Individual pieces of information, previously seen as substantially 
without implications, when interpreted and integrated revealed a 
striking picture of the patterns of minority-student success and 
failure. 

A second basic method of overcoming faculty resistance is to 
involve faculty directly in the design of the assessment process. 
Faculty involvement is strongly apparent in the case material 
presented earlier. Alverno's assessment process is explicitly based 
upon the cu*.*icular dimensions of competency previously 
developed by the institution s faculty. And at UTK, assessments 
cf general education were substantially aided by a series of faculty 
seminars on the ACT-COMP examination, the basis for data 
collection on general education. The positive tone set by these 
seminars was particularly important to the project's success. 
Seminar leaders were careful to point out the many limits of the 
particular measurement approach used and to enlist faculty 
interest in making careful, creative, nonjudgmental use of the 
results. 

Other examples stress the importance of involving faculty 
in the interpretation of assessment results. Spoon River College 
(SRC), a small rural community college, made student-assessment 
results the centerpiece of annual day-long faculty in-service 
retreat days. Explicit findings about student abilities, enrollment 
behavior, perceptions of the institution, and postgraduate place- 
ment were used to structure discussions of how to better retain 
particular populations of students. Similarly, both Mount Hood 
Community College (MHCC) and Towson State University 
(TSU) have effectively used campuswide student-success com- 
mittees to systematically review the implications of institutional- 
assessment information. |n all such cases, broad participation has 
produced better and more sensitive insights into the implications 
of particular assessment -results. More importantly, involvement 
has had a positive impact on the attitudes of many faculty toward 
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instructional issues. Many now report that, as a result of the in- 
formation provided, they are more sensitive to student concerns 
and more prone to question previously unexamined elements of 
the curriculum. 

In short, the primary method for overcoming faculty resis- 
tance to explicit institutional assessment is to make faculty full 
partners in the process from the outset. It is, after all, in individual 
classrooms that assessment information will be either used or 
forgotten. 



Many objections to establishing a comprehensive program of 
institutional -assessment center on the costs associated with doing 
so, The primary argument is that useful, accurate information 
on student achievement and instructional effectiveness is not 
currently available on most campuses and that providing such 
information entails establishment of an expensive data-collection 
"superstructure" largely divorced from academic affairs. Most 
faculty and many administrators are understandably reluctant to 
support the diversion of considerable resources from the main 
business of teaching and doing research. 

Admittedly, assessment efforts such as Alverno's can be 
expensive and require a careful look at institutional priorities. 
Alverno's budgetary commitment to assessment is backed by a 
faculty committed to assessing student learning as a developmen- 
tal process,^ well as to the more traditional program-evaluation 
goal of assessment. And, clearly, institutions should attempt 
efforts of such magnitude and direction only when there is a 
strong collective mandate to do so. For NMSU and UTK, the 
issue of resource investment is a good deal less clear-cut. NMSU, 
for example, maintains its comprehensive assessment program 
with the investment of relatively few additional institutional 
resources. The use of commonly available assessment instruments 
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such as the ACT Assessment, the ACT-COMP, and standardized 
student surveys have made the data-collection effort at NMSU 
unusually efficient. At UTK, moreover, the primary expense has 
been in providing administrative coordination for the assessment 
project— particularly in the provision of technical assistance to 
departments and schools wishing to design their own instruments 
, to assess student progress in the major field. 

In many cases, development erf an initial . institutional- 
assessment process involves minimal additional expense because 
institutions already collect a great deal of information about 
students. Indeed, one part of Alverno's triangulated model con- 
centrates on data already collected. One major intent of the 
recently completed NCHEMS/Kellogg Student Outcomes Pro- 
ject was to enable institutions to make more effective use of 
existing information about student outcomes. In seven public 
institutions, this initial assumption of the project was strongly 
confirmed (Ewell 1984). 

Scattered student-assessment data, however, is rarely in a 
form that makes it immediately useful. Different kinds ot data are 
collected by different offices for different purposes. It is often a 
matter of considerable effort both to inventory the data available 
on a large and complex university campus and to coordinate 
future data-collection efforts more effectively. Institutions that 
have undertaken these processes, however, have generally found 
the exercise to be of value. At MHCQ for example, campus wide 
assessment included placement testing, student follow-up 
surveys, and assessments of the effectiveness of developmental- 
education programs. Each of these data-collection efforts had 
been previously developed by a different campus unit, but their 
collective implications for student success had not previously 
been considered. At NCSU, a similar pattern emerged. Data on 
black student behavior was already routinely being collected by 
many different academic and student-affairs offices; integrating 
the findings jf these multiple existing efforts involved minimal 
additional expenditure. 
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Such uncoordinated student assessment already consumes 
many institutional resources As Asttn points out (1984a) f fhe ma- 
jority of colleges and uni vestries teutirtely collect information on 
srudent aptitudes and achievements to screen and place studies. 
But the fact that different instruments sre used at aiSer^nt times - 
generally precludes measurement of su*deiif change over rime. 
Some of the most effective institution^ assessments make consid* 
erahle use of this built-in pretest. NMSUV value sdded approach, 
for example, relies upon th* assumptions th**r many students will 
a!rmK have taken the ACT •Assessment and th« many students 
Umnd for j^i'iosc ^hooi will already want to tak*- the GRE ami 
professional ?pti:udc or cemik«;ion tests as the GMAT, LSAT, 
or NCTTK. At * IFK, moreover, approximately *wo-thi:ds of the 
departments thai % asa«*sed achievement in the major n<\d <:ould 
also make <ise of exiting *fandaruir*vJ tcs* instruments. An-i at 
Aiverno, *tuden? performance at a f^ven abMu; level is a built-in 
jjne'est foi tne r.rxt. 

In sh^rr. ii«vc toping a l^mpusw ide assessment process is cei- 
tamly not a ccstics* activity. Msnv of the examples c:r**d have 
relied upou some degree of rxternal funding to accomplish their 
goals tor, in the case of IJTK, c**.Tnal incentives in the form 
of performance funding). In the majority of cases, however, the 
expenditure has been relatively mode*:. The average institutional 
invrsrment in the NCHEMS/Keliogg project over a three-year 
penc*! was approximately $35,000, matched by approximately 
*2 1,000 in external funds and services. From a marg nal point of 
view, fr*> equivalent investments could have voided a higher 
return 



Objections Base d on Lack of Administrative Incentive 



Finally, there <hc objection that most colleges and univer- 
sities are simply m« strKtun-J to take advantage of comprehen- 
sive assessment ^formation. In most institutions, traditional 
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academic Organization is decentralized, if not fragmented, j 
Responsibility for overall student success and development is < 
scattered throughout many offices among many individuals, each J 
of whom deals with a particular "piece" of the individual student. 
More importantly, institutional reward structures, particularly in 
the public sector, are centered more around productivity than 
around effectiveness. These two structural facte about most 
colleges and universities constitute a formidable challenge to 
information-based qualitative improvement. If the fundamental 
incentive structure of the institution cannot he modified to 
reward innovation and achievement as revealed by assessment 
information, then comprehensive data collection will indeed 
become what its critics fear— an administrative superstructure. 

In the case of Alverno, this outcome was avoided by a com- 
plete remaking of tlw institution's curriculum. Together with the 
new curriculum came a new way cf looking at institutional 
success and failure— one shared by faculty and administration 
alike. Few institutions will want or be in a position to undertake 
so radical a transformation. Many examples of less radical 
approaches are available, however. Seme of these rely upon pro- 
viding successful unit? with modest additional resources. Others 
rely upon building assessment information into a formal structure 
6f subunit review and accountability. Finally, an increasing 
number are being imposed on institutions from without— by state 
boards, accrediting agencies, and government bodies. 

Certainly, providing departments and units with direct in- 
centives to increase performance or to collect and use additional 
assessment information is a promising approach. Indeed, from a 
state perspective, this approach constitutes the original rationale 
of the Tennessee Higher Education Conmission's experiment in 
performance funding. This approach has also been effectively 
implemented at the campus level. The UTK assessment project, 
for example, provided limited "mini-grails" to departments and 
schools willing to undertake pilot projects in assessing student 
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achievement in the major field. Montana State University (MSU) 
makes extensive use of such mini-grants, both in providing 
limited funds to collect and use information on student attitudes 
and achievements and in funding projects designed to improve 
faculty vitality In most cases, the funds provided have been 
relatively modest and the primary effect has been to provide a 
visible symbol through which to recognize outstanding or in- 
novative effort. Although symbolic, such recognition has been 
considerably sought after. 

A more common approach has been an attempt to build 
student-assessment information directly into the institution's 
formal accountability structure. For example, at SUNY-Albany, 
information on student placement in jobs and in graduate 
schools, and student ratings of the effectiveness of departmental 
instruction in their major, are distributed to each department as 
pan of the annual planning/budgeting process. Departments and 
schools are asked to address 'his information formally in the 
course of preparing their five-year plans and budgets. While this 
procedure, now two years old, has not produced notable changes 
fin existing patterns of resource allocation, it has often signifi- 
cantly altered campus perceptions of what top administration 
feels is important. Several department heads at SUN Y- Albany, 
for example, have structured department meetings in response to 
this information; this has resulted in curriculum changes. 

In many cases, the focus of such efforts has been increasingly 
formalized in a process of academic program review. At four of the 
seven institutions participating in the NCHEMS/Kellogg Stu- 
den r Outcomes Project, for example, academic program-review 
processes were instituted or substantially modified. In each of 
these cases, informauon on student attitudes and achievement by 
department was collected and reviewed, and explicit recommen- 
dations for improvement were sought* Like the SUNY-Albany 
experience, these program reviews rarely produced major shifts ir 
institutional activity. Very rarely was a program substantially 
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modified or eliminated. However, a prime virtue of the process, 
reported by both faculty and administrators, was that it stimu- | 
iated increased attention toward the curriculum and toward 
student learning. 

A common theme in all of these efforts is the creation of a 
concrete set of incentives— both material and moral— to prompt 
teaching faculty and department administrators to respond 
explicitly to assessment information. And indeed, in many cases 
department chairs have reported that explicit assessment data 
allows them to take action on what had previously been mere 
suspicion. At SUNYAlbany, for example, the chair of the 
English department wa* ?*ble to use student data as authority to 
tighten standards in the curriculum— an action previously tabled 
for lack of evidence. 

Such incentives are simultaneously more powerful and more 
threatening when implemented from outside the individual insti- 
tution. Growing numbers of funding and accountability bodies, 
particularly in the public sector, are developing outcomes stand- 
ards to ar^iy to institutions under their jurisdiction. Certainly, 
the mos; highly developed of such programs is the performance- 
funding initiative in Tennessee. Unique in many ways, this 
program is probably most unusial in that it provides positive 
rewards for institutional performance. Such rewards, amounting 
to substantial resources in the case of major institutions, have 
proven to be powerful levers for the development of exemplary 
institutional-assessment programs such as that at UTTC. 

Few developing ext *rnal incentive structures, however, have 
been so positive. Mere typical has been the approach of Florida in 
mandating specified levels of student achievement for enrollment 
in the upper division. Questions about student achievement and 
other outcome measures have increasingly been asked of public 
institutions in the "ourse of state-level, legislative or executive 
audits. While such procedures undoubtedly provide incentives' 
for institutional improvement, they also contain the potential for 
considerable misuse. 
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The external pressures generated by such state-level programs 
as in Tennessee or Florida have their best results in the leverage 
that they provide institutional administrators ho want to build 
. a case for assessment on their own campuses This leverage has 
been admirably exploited, for example, in the development of 
department-level assessment processes at UTK in Tennessee and 
in the development of an academic program-review process at St. 
Petersburg Junior College in Florida. 

It is important to stress, however, that proof of the effective- 
ness of such externaUyinandated standards ultimately lies in the 
climate these standardscreate for institutional self-examination. If 
external standards are too heavily imposed, institutions will be 
driven into a defensive posture. By its very nature, such a posture 
reverses the intended incentives. Response to external pressure # 
becomes protective and bureaucratic— and information gathering 
becomes an activit/ that faculty either eschew or treat with 
cynicism. The inevitable result will be minimal impact on the 
classroom or curriculum. * 

Perhaps the most effective role of external incentives is to 
induce institutions to undertake certain processes rather than 
demanding a particular level of performance. The outcomes 
criterion for institutional accreditation currently under review 
by the Southern Assochtion of Colleges and Schools provides an 
example of such an incentive. The essence of the criterion is that 
institutions should give proof that they engage in significant, 
information-based assessment of the outcomes they produr 
There is no requirement for use of a particular set of outcomes 
dimensions or measurement procedures. Such questions arc quite 
properly left within the prerogative of the individual institution. 
^ All that is required is that the institution, take self-assessment 
seriously and communicate the results. The three objections to 
outromes-based assessment described above constitute a formi- 
dable collective obstacle to developing systematic procedures for 
institutional self-exarni nation. But they can be overcome. Perhaps 
the most important lesson of efforts such as the NCHEMS/ 
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Kellogg project is the value of making a beginning. Indeed, it may 
be that the greatest payoffs of an institutional-assessment process 
lie in its initial stages, when the actual informational basis of the 
assessment is at its weakest. The significance of the process lies not 
in the facts it uncovers but in the questions it raises and in the 
resulting dialogue it provokes. In many of our institutions, such a 
dialogue will, in itself, be an accomplishment of quality. 
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CHAPTER 7 



Four Common Themes 
Of "Information for 
Excellence" Programs 



As emphasized thrr jghout this book, every successful insti- 
tutional self-assessment process must reflect the particular 
culture and character of the campus on which it is located. There 
is consequently no single recipe for success in, a venture of this 
kind- Nevertheless, there are several themes apparent in such 
efforts, and those who seek to implement such program* at par- 
ticufar institutions are well advised to examine them. 



\ Focus on the Curriculum 



The first major theme of successful self-assessment efforts is an 
explicit linkage with the institution's curriculum. Findings of the 
assessment must be directly relevant to current curricular issues 
and disseminated in such a manner that those responsible for cur* 
riculcm review and development can immediately use the results. 
Often tlvs v/ill mean that the administrative center of rhe assess- 
ment resides in the office of the institution's chief academic-affairs 
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officer. This is the case at Northeast Missouri State University 
(NSMU), At Alverno and the University of Tennessee, Knoxville 
(UTK), the special units charged with the responsibility for assess- 
ment have strong linkages with the forma! structure of academic 
affairs. The efficacy of a curriculum focus has been strikingly con- 
firmed by institutional experience in the NCHEMS/Kellogg 
Student Outcomes Project. At NCSU, for example, the impact of 
the assessment project was at first quite limited because the 
project was centered in a student-affairs office. Despite the com* 
prehensiveness and quality of the data collected, few major 
changes were accomplished in the first two years of the project 
because the academic accountability structure of the institution 
was not directly involved. At the State University erf New York- 
Albany (SUNY Albany), Montana State University (MSU), and 
Mount Hoed Community College (MHCC), morever, the assess- 
ment effort was based in administrative offices of institutional 
research. While such offices brought cpnsiderable technical 
expertise to the assessment process, much of the challenge in 
these pro\xts was to build regular relationships with deans and 
department chairs, those responsible for shaping and evaluating 
the curriculum. Success in these three campus projects was largely 
a product of how well these relationships were established and 
maintained. 

These projects provide excellent illustrations of how explicit 
linkages can be built between those responsible for curriculum, 
and the information professionals who will likely be involved in 
any assessment project. As mentioned, regular department-specific 
data reports were supplied by the assessment or institution?? - 
research office to academic administrators at SUNY- Albany, 
UTK, anu MHCC. On each campus, the implications of ti»£ 
assessment data prompted departments to undertake modifica- 
tions in curriculum at the department level. Often, however, the 
interpretive role of the information professional proved critical 
in making administrators aware of the limits and the potential 
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of particular pieces of information. Indeed, one of the most im- 
portant impacts of regular formal communication between data 
providers and data users was to create an incentive and an oppor- 
tunity for a much more important pattern of ongoing informal 
contact. 

Examples of the kinds of curriculum impacts that have been ■ 
accomplished on these campuses are many. Some concern the way 
certain types of general or discipline-specific skills are taught and" 
maintained. At NMSU, for example, assessment has consider- 
ably increased the number of required writing and mathematical 
exercises students encounter in the curriculum— and value-added 
assessment results have shown gains in mathematical ability as a 
result. At SUNY-Albany, the English department undertook 
similar revisions oPcurriculum content after receiving informa- 
tion that their graduates, now in graduate ^programs or the 
workplace, wished that the faculty had been "harder on them 
when, the> were enrolled/ 1 

Other examples of curriculum impact have to do with the 
role of particular courses in the curriculum— particularly the 
introductory course in a given department, or key "bottleneck" 
courses in a particular program that all students must pass. At 
North Carolina State University (NCSU), for instance, assess- 
ment revealed that a major reason why black students were not 
successful in many technical programs was that they were enroll- 
ing in certain courses too early. In an attempt to move through 
the curriculum at a faster rate, they enrolled in difficult courses 
before they had folly mastered important prerequisite skills. This 
di c overy resulted in a more structured approach to the way 
students were advised and placed. Moreover, in a real estate 
program at MHCC f the order in which practice and theory 
courses were taught was reversed. Assessment data on the kinds 
of jobs students were getting after they left the program indicated 
a change; a shorter certification program was the result. 
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Many additional examples of curriculum impacts could be 
cited. But there are equally numerous cases where curriculum was _ 
not touched by an assessment process. The vast majority of these 
occurred at institutions where data collection was both physically 
and administratively distant from those responsible for curricu- 
lum development and where top administrators have created few 
incentives for the two to get together. 



Choosing the Correct Unit of Analysis 

A second theme of successful assessment programs follows 
directly from a focus on the curriculum. This theme emphasizes 
the importance of choosing the correct level, or unit of analysis, 
for campus assessment. Different institutions— depending upon 
their size, structure, mission, and student composition— can have 
vastly different patterns of human interaction. Such patterns, 
furthermore, are reinforced by curricular structures that encour- 
age or discourage common course taking by students in different 
programs. At some institutions^-particularly small, residential 
institutions— the entire campus may constitute the appropriate 
unit for assessment. At other institutions, -..ost learning and 
interaction may be concentrated at the level of the department, 
school, or program. And at still other campuses— particularly 
those such as community colleges with multiple constituencies— 
the most appropriate units of analysis may be the sevqral particu- 
lar and distinctive types of students inhabiting the institution. 

Often, of course, a single program will simultaneously ap- 
proach assessment on multiple levels. At UTK and NMSU, for 
example, general-education assessment is reported at the campus 
level, while eachj. department undertakes its own, department- 
specific assessment process. At SUNV- Albany, results of student 
follow-up surveys were used to produce campuswide issue reports 
on such topics as student-faculty contact and student career 
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orientation. The same data were also broken down by depart- 
ment to generate local discussions of their implications. Alverno 
College undertook specific assessments of particular professional 
programs to supplement their primary assessment focus on the 
institution as a whole. 

It is important to note that the success of these mixed 
strategies was strongly dependent upon the match between assess- 
ment and institutional culture. At Alverno, where a collective, 
integrated campus culture is present, campus-level results were 
effectively internalized and used. At SUNY-Albany and UTK, in 
contrast, the impact of campus-level assessments was extremely 
limited; results were more apparent when assessment concen- 
trated on individual departments. In such large, decentralized 
institutions, department chairs could often quite rightly claim 
that the results of campuswide assessments, however true, did not 
apply ro their students. 

This general point regarding applicability is equally telling in 
assessments of particular student groups. On mult [constituency 
campuses, it is important not to conduct assessment as though all 
students were alike. At Mount Hood Community College 
(MHCC), Spoon River College (SRC), and Towson State Univer- 
sity (TSU), for example, much of the assessment process consisted 
of developing profiles of different student "prototypes" based 
upon different demographic characteristics and patterns of 
behavior (Sheldon and Grafton 1982). At SRC, for example, five 
distinct types of students were identified. Each type was made the 
target of a different set of advisement, skills-building, and reten- 
tion strategies. Furthermore, all subsequent assessment reporting 
included distinct breakdowns and analyses for each of these five 
basic student populations. 

The must meaningful assessment takes place at the level 
where students actually live and learn, where actual campus 
interaction takes place. One of the most interesting elements of 
the assessment process at MHCC, for example, is a computer- 
generated course report received by each faculty member for each 
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course each term. This report contains standard enrollment 
information (such as major and level) on each student, but also 
presents data on student intentions, previous GPA, skill levels, 
previous education, and many other elements. The result is that 
faculty, particularly in large diverse classes, are much better 
acquainted with the needs and characteristics of the students they 
are facing. 



The Importance of Comparison 

Regardless of the unit of analysis chosen, effective compari- 
son among units of like type should be the analytical focus of any 
assessment. Many types of comparison, of course, are possible, 
and the choice of which to use is largely a matter of institutional 
goals and char; -^eristics. Comparisons of performance at the 
same institution over time are particularly valuable in situations 
where institutional goals are seen as unique ami, consequently, 
difficult to compare to other institutions. Peer comparisons, 
however, are particularly valuable in assessing or demonstrating a 
particular institution's competitive strengths and weaknesses. 

The institutions observed here have used many types of com- 
parisons. At Alverno, specific identified abilities are a central 
focus of the learning environment and may not be shared by all 
institutions. Nevertheless, the choice of which assessment in- 
struments to use was influenced by the availability of life-span 
developmental norms from other institutions. At NMSU, o m- 
parisons over time are used to chart institutional progress in value 
added, but departments are also evaluated in terms of their 
students' performances against national norms on standardized 
achievement tests. At SUNYAlbany and UTK, comparisons 
among departments are made on a number of standard indicators 
of student satisfaction and placement, while the same data are 
also analyzed campuswide by student type. In all cases, the 
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richness and range of comparison has considerably enhanced the 
utility of assessment data to stimulate action. 

Part of the value of a comparative orientation is that it allows 
effective use of data that are much "softer" than those to which 
administrators are often accustomed. Much of the data available 
on student outcomes is— as critics are quick to point out— 
methodologically limited. Questionnaire or test data may be 
biased in unknown ways due to the wording of particular ques- 
tions, the characteristics of the sample, or any number of other 
factors. Hence, profiles of results are much more useful than single 
assessment indicators, and such profiles are most effectively 
presented comparatively. To know that 76 percent of the student 
body in a given institution is dissatisfied with advising, in itself, 
tells you little. However, knowing that 45 percent of the same 
sample last year gave the same response, or that the respective 
figures for English and for business students are 89 percent and 57 
percent may tell you a great deal. 

The most useful single stt of comparisons are those that can 
be made among the individual academic units or programs within 
an institution. li> .he NCHEMS/Kellogg Project, mechanisms for 
comparing departmental performance on student outcomes— 
whether freestanding or as a part of an academic program-review 
process— were uniformly effective in generating dialogue about 
departmental curricula and instruction. 

In their initial stages, it is important to note, not all such 
dialogues were cordial. Department heads and faculty were quick 
to point out the deficiencies of any comparisons on which they 
tame up short. Still, all sides realized the value of a process that 
could inspire a dialogue on program effectiveness— a dialogue that 
otherwise would never have taken place. Over several years, 
department chairs in all participating institutions began to 
recognize the opportunities that the process provided for 
highlighting and communicating their own needs and stories. 
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The Importance of Involvement 

All three of the above themes are subsumed in a fourth: the 
importance of broad participation in the construction and opera- 
tion of an information-based assessment process. Promoting 
faculty involvement, by whatever means are at hand, is an impor- 
tant tactic in establishing a campuswide institutional-assessment 
structure. But we mi not forget that faculty involvement it also 
the primary end of such a structure. Indeed, explicit institutional- 
assessment information is of ultimate utility only insofar as it 
induces faculty and administrators to regularly revisit and discuss 
basic educational issues, 

.\ Astin has effectively argued, a theory of student involve- 
ment provides a comprehend ve framework for organizing many 
discrete findings about why students learn and persist in college 
(Astin 1984b). According to Astin's account, it is the sum ratal of 
involvement in the educational process that is important, not its 
particular source or mix for an individual student. Some students 
will seek a purely academic engagement in college iife and wil 1 
succeed on this basis. Others will increase their "time on task" 
with extracurricular activities, such as work-study assignments, 
orga.'-zed student activities, or informal contacts with others in 
the - allege environment A primary implication of the theory is 
its focus on student time as a resource Learning will be enhanced 
to the degree students can become (or can be induced to become) 
more conscious both of the ways they spend their time and the 
consequences of particular time investments. 

If this view of effective learning is true for our students, wc 
should explore its implications for ourselves. As teachers and 
administrators, we too must seriously examine our "time on task" 
in the business of promoting learning. One major virtue of an ex- 
plicit institutional-assessment process, like the explicit curricular 
struc tures we construct for ou$ students, is that it channels our at* 
tent ion toward matters we might otherwise be inclined to forget. 
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The experiences of many colleges ami universities highlight 
this contention. Common to all institutions participating in the 
NCHEMS/Kellogg project was the role of explicit assessment in- 
formation in making p e discussions of institutional mission 
and effectiveness in the m place* Indeed, in some institutions, 
this general outcome was seen as having greater lasting value than 
any particular implications of a given data-collection effort. 

At DTK, for example, some of the most important 
department-level impacts of assessment were in those depart* 
merits that developed their own assessment instruments. The ab- 
sence of a proven standard test of achievement in the major field 
was first seen as a major weakness. And in fact, the assessment 
instruments designed by faculty to fill the gap were admittedly 
crude compared to the standardized examinations available to 
other fields. However; the process of developing such instru- 
ments, regardless of their elegance, induced faculty to re-examine 
curricular objectives ami methods. And the actions resulting in 
such departments often proved a good deal more comprehensive 
than in those receiving assessment information from an external 
source. 

A similar dynamic was also apparent in general-education 
assessment at SUNY Albany. To implement a new requirement 
for distribution-based general education, a faculty committee 
spent considerable effort in defining and operationalizing six 
dimensions of general education. While in the last analysis these 
dimensions differed little from those developed at many other 
institutions, the processes of articulating them and trying to 
decide how to assess them were of considerable independent 
value* 

The use of campuswide committees to foster suchiiiscussion 
is one element of involvement relatively easy to accomplish. An 
important requirement for institutional participation in the 
NCHEMS/Kellogg project was that institutions constitute a 
broad-based committee to interpret available student- Assessment 
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information and develop recommendations as a result. The con- 
tribution of such committees to the ultimate success of the project 
cannot be overstated. At NCSU, for example, committee discus- 
sions immediately revealed areas where particular academic and 
student- service units were working at cross-purposes, or where 
units were unnecessarily duplicative in collecting information 
about students. 

The effectiveness of such committees, however, depended 
upon more than simply creating the opportunity for people from 
different parts of the institution to talk about students. The* 
fundamental difference between committee discussions within 
the context of the assessment project and previous institutional 
experience was that discussions in the context of assessment were 
focused and were based upon explicit information. The presence 
of concrete, empirical assessment data repeatedly allowed 
previously open and circular discussions to come to a conclusion. 
Interestingly, this was even the case when the assessment data 
provided were of limited quality. At TSU, for example, it took 
over a year to produce program retention data of acceptable 
i,j*»lity, but committee discussions around preliminary data 
remained refreshingly concrete. Not only were actions taken as a 
result, but the better data available later in the assessment process 
revealed these to have been the proper actions. 

In short, comprehensive institutional assessment provides 
one important structure to promote the kind of involvement in 
learning issues that we are likely to forget in the fragmented, 
means-oriented world of most higher-education institutions. 
Perhaps the most eloquent statement of this result comes from a 
faculty member at UTK: "Thanks for giving us the opportunity 
for doing something that we have wanted to do for a long time." 



92 



93 



T 



CONCLUSION 



Institutional 
Assessment and 
Self-Renewal: 
A Developmental 
Metaphor 



he recently released report of the Study Group on the 



A Conditions of Excellence in America i Higher Education, 
sponsored by the National Institute of Education, emphasizes the 
need for explicit feedback and assessment to effect needed inv 
provements in undergraduate instruction (NDE 1984). This need 
is based, in part, upon what we know about the effectiveness of 
active involvement in the learning process. Students learn better 
when they are actively engaged, when they are challenged, and 
when they are continually made aware of both the process and 
prospect of their own development. Administrators, teachers, 
and ultimately institutions themselves are no different. Improve- 
ments will result only when expectations are clear and when per- 
formance relative to these expectations is systematically assessed. 

At the same time, the Study Group suggests, the landscape of 
higher education has changed dramatically. Institutions have 
grown in size, complexity, and mission. New clienteles with new 
needs, resources, and learning styles have been added to the tradi- 
tional 18- to 2 1-v ear-old undergraduate student population. New 
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programs and curricula, often occupationally or professionally 
oriented, have supplanted the traditional liberal arts as study 
'areas of choice among undergraduate students. And, finally, 
these programs and curricula are increasingly delivered in new 
ways, often involving new technologies. Admittedly, through 
high selectivity, substantial resources, and a commitment to 
undergraduate teaching and learning, many individual colleges 
and universities have remained substantially unaffected by these 
massive environmental changes. But the majority of our institu- 
tions have experienced such changes— and have become larger, 
more diversified, and more formally structured as a result. In the 
face of such changes, what is to be done to revitalize undergrad- 
uate education? The answer, in part, depends upon our view of 
this transformation itself—whether we view these changes as 
reyersible or inevitable, and as creative or destructive. 

One of the great conceptual debates of sociology took place at 
the end of the nineteenth century and concerned die nature and 
virtue of traditional society. In Gemeinschaft and GeseUschaft 
(1887), Ferdinand Toennies argued that traditional society was 
based upon a web of implicit relationships that were fused and 
self-reinforcing, Close contact, shared values, and the unity of 
workplace and family life all contributed to a stable and inte- 
grated culture. Advancing industrialism caused a breakdown in 
this structure, with a resultant erosion of traditional virtues and J 
values— a prospect Toennies viewed with alarm. Emile Durkheim, - . 
on the cher hand, argued in "On Mechanical and Organic 
Solidarity" (1905) that the emerging pattern of social relations 
created its own kind of solidarity, one based upon explicit struc- 
tures, information exchange, and role differentiation. Far from 
eliminating old virtues, such a structure allowed them to be 
perpetuated and elaborated in new, more fruitful ways. At stake 
in this argument, as noted by Laslett's The Worid We Have Las* 
(1965), is more than a faulty account of how things have changed. 
Our whole view" of ourselves is altered based upon our belief that 
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we have lost some more humane, much more natural pattern of 
rf relationship than that which modern society has to offer. 

Arguments of similar kind increasingly seem to characterize 
discussions of American postsecondary education. Certainly, 
there exists no shortage of commentators to lament, in Laslett's 
words, "the world we have lost." Indeed, for many, the solution to 
achieving excellence in our current array of diverse institutions 
lies in a reaffirmation of traditional virtues; strong purpose, high 
standards, and a commitment to character typical of the small 
private institution of an earlier age (Martin 1982). 

.Such virtues are indeed important, ami much can be learned 
by reexamining them. The operational question, however, is how 
such a reexamination can, in fact, take place. To expect a faculty 
in the complex environment of a large public research university, 
a multiconstituency community college, or a public regional 
service institution to practice such virtues simply because we tell 
them that such activities are valuable and traditional is surely 
not enough. We are not going to change the structures of what 
now constitute the majority of our institutions through pro- 
nouncement or exhortation- We need, rather to build explicit 
administrative mechanisms, suited to the reality of our current 
institutions, to induce faculty and administrators to do what in a 
different setting they once did naturally. 

Explicit, outcomes-oriented, institutional- assessment pro- 
cesses constitute one such mechanism, and one much more easily 
attainable than many now believe. Such assessment processes, 
like all formal accountability processes, have their dangers* In 
an era characterized by shrinking resources and greater public 
demands for short-term effectiveness, voluntary assessment on 
the pan of colleges and universities constitutes an act of consider- 
able courage. Far from being irrational, institutional resistance to 
publicly evaluating institutional effectiveness is based on a real 
and historically justified apprehension that such processes will be 
misused by those outside the higher-education community. As 
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Durkheim warned about the emergence of explicit laws and roles 
in modernizing Europe, accountability structures can be oppres- 
sive as well as integrating. 

But we may no longer have a choice. Public concerns for 
quality in higher education are real and persistent. Answering 
them will require that all institutions contribute The challenge of 
the next decade will be to balance such concerns, legitimate 
though rhey may be, with a sensitivity for institutional differences 
ami with a respect for participatory, faculty-centered decision- 
making. Explicit, institution-specific assessment processes can 
achieve such a balance, but only if they are implemented slowly, 
carefully, and with proper and constant reference to the ends of 
the enterprise— the vitality of the academic community and the 
students who inhabit it. 
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